Gentoo Weekly Newsletter: December 5th, 2005
GCC 3.4.4 stable on x86
The compiler is a central component of every Gentoo system, so unlike other
packages, the developers are rather careful when updating it. Finally gcc 3.4.4
has made it to the stable branch on x86, affecting a large number of users.
on the developer mailing list links to the upgrade
guide containing the steps necessary for the upgrade. It is important to
rebuild some packages before unmerging the old gcc version or the system may break.
According to the
thread in the Gentoo forums, some minor problems occured on some systems
due to a small bug, but once it was fixed the upgrade worked fine for most of
If you experience problems or have general questions, visit the forums
or #gentoo on irc.freenode.net, and please report
bugs on http://bugs.gentoo.org.
Power.org opens free developer membership
Gentoo PPC developer Luca Barbato was the
first to hop aboard a new initiative from Power.org,
a collaborative industry consortium dedicated to "developing, enabling and promoting
Power Architecture technology innovation" and representing a wide range of semiconductor
and electronics organisations active in the field. Founded by IBM, Power.org tries to open
itself to the community of POWER, PowerPC and related architecture development, and counts
Gentoo sponsor Genesi among its recent additions to contributors and supporters. Developers
who become members of Power.org get access to technical resources at the member-only area on
the website, and to technical subcommittee output. You can register at the Power.org website.
GWN looking for contributors
Looking at this week's unusually rich issue of the GWN, you wouldn't suspect that we sometimes
have serious trouble getting our weekly edition out in a timely and accurate fashion. The
author list on this page has dwindled down to less than a handful of regular contributors. If
you would like the GWN to continue to provide you with information from the Gentoo project, all
its outer layers in the user communities, commercial spin-offs or fun stuff emerging from
the greater Gentoo realm -- then we'd like to encourage you to join us on the authoring team.
Many of our current news sections could use a helping hand, for example everybody's favorite
GWN chapter, the Tips & Tricks, is particularly in need of input from as many sources as
possible. Send a message to firstname.lastname@example.org
and tell us for which of the existing sections you'd like to contribute articles, or maybe come
up with entirely new ideas, offer proofreading (very much appreciated), invent new and
exciting techniques to automate some of our scripting, or indeed whatever else you feel up to.
100k users registered at the Gentoo Forums
Three and a half years after the creation of the Gentoo forums we hit 100,000 registered
users early on 16 November morning when Blindside350 registered:
Figure 1.1: Screenshot dated 16 November 2005
A forum thread
from a very distant past has been reanimated for the occasion.
Interview with Bryan Green at NASA's Advanced Supercomputing division
Gentoo made a public appearance at NASA's visualization wall last month at the
Supercomputing 2005 conference.
Thousands of visitors were able to see the status and results from NASA's Columbia
supercomputer, through a visualization system running Gentoo. This week, Bryan Green
tells us more about this system and how Gentoo is being used in the NASA Advanced
Tell us about yourself and your work with Columbia (along with a few words
about Columbia for those who are not familiar)
I am a contractor working at the NASA Advanced Supercomputing (the NAS,
formerly "Numerical Aerospace Simulation") facility, at NASA Ames Research
Center in Mountain View, CA. I am an employee of AMTI. I've been a member of
the NAS facility's scientific visualization group since 1997, when I graduated
from UC Santa Cruz. When I started, we were using SGI Onyx systems for
visualization, but the emergence of NVIDIA, and the pressure of the PC gamer
market on the graphics card industry, ultimately made it possible to move to
commodity-based linux systems with high-end NVIDIA graphics cards. So we have
the gamers to thank for saving us hundreds of thousands of dollars on graphics
hardware. My group's home page is here:
The Columbia supercomputer, a 10,240 processor SGI Altix system running Linux,
was built by the NAS in 2004. It uses Intel Itanium2 processors, and is
divided into 20 nodes with 512 processors each. Each 512-processor Altix node
is a Single System Image (SSI) with 1 Terabyte of shared memory. More
information about Columbia and the various types of science being performed on
it can be found here:
While Columbia itself uses a Suse-based distribution (with SGI modifications),
the NAS division's security group uses Gentoo exclusively for infrastructure
and special projects. They even have a three-node Mosix cluster. In total,
they have about 30 Gentoo systems deployed. If anyone is interested in
more information about this, they can contact Derek Shaw.
Since Columbia was built, my group's work has primarily involved providing
visualization support to Columbia users. Our focus these days is on tapping
directly into the simulation codes themselves, and extracting data as it is
produced. We have our own 16-cpu-node "chunnel" plugged into Columbia. For
each simulation timestep, we extract the computed data via shared memory, ship
the data over infiniband to chunnel, and distribute it to our visualization
cluster. During this past hurricane season, we have been using this technique
to generate movies from runs of fvGCM, the finite-volume General Circulation
Model which has been used in hurricane forecasting. This was one of the
applications we were demonstrating at SC05.
On to the Columbia monitor. What all are you able to visualize and see on
these screens? What else do you do with this visualization cluster?
We refer to our grid of LCD screens backed by a cluster as a "hyperwall". As
opposed to a "powerwall", which typically displays large images, our intent is
to visualize high-dimensional datasets by allowing the use of different tools,
viewpoints, and parameters to display different aspects of a dataset
simultaneously and in concert. We have a 7x7 hyperwall installed at our
facility and a 3x3 hyperwall which serves as our "portable": the one you see
in the picture. The hyperwall has been used for a number of applications
including Computation Fluid Dynamics (CFD), quantum physics, astronomy and
astrophysics, and atmospheric and oceanic modeling.
The 7x7 is still running Fedora Core 2. It is our original hyperwall, and
whether/when/how it gets updated and/or converted is subject to debate.
The Columbia monitoring tool is a program for monitoring system resource
utilization on one or more Columbia nodes. With the hyperwall, we can monitor
all 20 Columbia nodes simultaneously with plenty of resolution.
Our ultimate goal is to combine system resource monitoring with code
instrumentation to get a holistic view of a specific simulation's behaviour,
performance and system utilization, ultimately for optimization purposes.
Our division provides "full-service" support for High End Computing (HEC)
applications -- porting and scaling, optimization, and visualization.
Figure 2.1: Statistical information
Most of the statistical information is gleaned from the proc and sysfs
filesystems. Three monitoring components are shown in the picture: cpumon,
pagemon, and linkmon. Cpumon visualizes cpu statistics gleaned from
'/proc/stat', as well as showing cpuset allocations. Pagemon visualizes the
non-uniform memory usage -- memory in a 512-processor node is divided into 256
blocks with 4 Gigabytes each, with statistics gleaned from
'/sys/devices/system/node/node*/meminfo'. Linkmon uses the "Performance
Co-Pilot" tool to get NUMAlink statistics -- Columbia's communication backbone.
The charts in the upper left screen show cpu and memory statistics for the
64-processor Columbia frontend node (bottom), and the vis group's 16-processor
node previously mentioned (top). Each of the 5 screens arranged in a cross is
subdivided into quadrants, with each quadrant showing statistics for a single
Columbia 512-processor node. A set of unhappy faces means either a node is
down, or the monitoring daemon is not running on that system. The set of
three views on the lower right screen is just a blowup of the Columbia node on
which the production fvGCM runs are performed, from which we produce our
What do the colored boxes, bar graphs, and lines represent?
The grid of colored boxes (cells) shows cpu utilization. Each cell represents
one cpu. The color codes are as follows: black=idle, blue=user, red=system,
cyan=iowait, yellow=soft-interrupt, orange=hard-interrupt. Cpusets are
represented by colored outlines around groups of cells.
The red bar graphs show memory usage. For a 512, there are 256 vertical bars
representing each of the 4-Gigabyte blocks of memory. The height of the red
bar indicates the percentage of memory allocated.
The line graph shown below the cpu and memory charts is a representation of
the NUMAlink topology for a 512-cpu node. To quote our SC05 poster:
"Dual-plane fat tree NUMAlink interconnect topology, with activity shown in
magenta between the 2 nodes within the green squares in a C-brick, and between
the nodes and the routers represented by cyan squares; level 1 and level 2
metarouters are indicated by the yellow squares." Note that the term "node"
here refers to a portion of a C-brick, not to an entire 512-cpu "node".
Magenta=activity, blue=inactivity, gray=unreadable.
Give us a few details about how the 3x3 screen wall works. How do you
The hyperwall is a fairly simple cluster. Each node is a dual-CPU, dual-core
Opteron system with 4 Gigs RAM. Each node has a basic Gentoo install on it,
with Xorg and Xfce, and of course the NVIDIA video driver for OpenGL. We use
NFS for shared filesystems. We have a set of scripts for things like
launching programs on the nodes. Nodes can be selected by their screen's two
dimensional position in the grid, using perl regular expressions to select
groups of nodes -- so the cross pattern used in the picture is described by:
'CROSS = N1. N.1'. We have a tool called 'hyperX' which allows you to drive
the cursor on all the screens, or subsets of screens, simultaneously from the
master console. It works using the XTest protocol. I have also written a
C++-based distributed object and event architecture that is used for driving
some of the applications on the wall. This is the underlying framework used
by the Columbia monitoring tools.
Figure 2.2: Statistical information
One application approach is to launch a copy of an application on each screen
and drive each instance simultaneously. Each instance of the app has
different initial conditions, for example the data field being mapped onto a
particular geometry, or a variation in the colormap. The apps are driven
either via their GUI, using hyperX, or via a master control panel that
controls the apps with app-specific protocols. The advantage of hyperX
is that virtually any application can be used unmodified.
Another approach involves more intercommunication between the apps running on
the nodes, so that the view on one node influences the view on another in some
way. For example, constraints in one set of dimensions may be set visually in
one display, and the mapping of those constraints into other dimensions may be
seen in the other displays.
Why did you pick Gentoo for this setup?
I simply wanted to try installing Gentoo on a cluster. I first installed
Gentoo on my work laptop, and after becoming familiar and comfortable with it,
I installed it on my workstation. Until then, my group's systems were
exclusively Redhat/Fedora. Gentoo reignited my love for Linux, and it made
system administration interesting and enjoyable for me. I started getting the
idea that it would make a great distribution for a cluster, and when we built
our first mini hyperwall (the first 3x3), I had partitions set aside for later
installing Gentoo alongside Fedora. After a good long wait, I finally did the
deed, and it actually went really well. I then installed Gentoo on our second
3x3 about a week before SC05... well, actually, I cloned the original 3x3, and
then just tweaked and updated. If you want a rational motivation for the
install, besides simple experimentation, it was because I really like and
depend on Portage for convenient access to a diversity of applications and
keeping certain packages up-to-date, like GTK. I was always having trouble
with installing non-standard packages on Fedora, usually because no one had
bothered to make an RPM for them, or I got stuck wrangling with yum.
What are the big advantages of using Gentoo in this particular setup? How
about the disadvantages or areas that need improvement?
For one thing, the ability to find useful information, documentation, and
community support is very valuable and something that I always felt was
lacking from Redhat/Fedora.
I setup my first Portage overlay to deal with an issue involving our newest
3x3. The NVIDIA video driver had major stability issues on our hardware. We
got NVIDIA to identify and fix the problem, and they provided us with a patch.
I copied the latest nvidia-kernel ebuild into a Portage overlay and updated it
to apply the patch. Its great to be able to have our custom-patched graphics
driver reside in the package management system -- and it was really easy to do.
Portage offers a number of advantages over binary distributions. Access to a
diversity of packages is one. Even browsing/searching Portage is a great way
to learn about what tools are out there. The ability to be selective about
what versions of packages get installed has come in handly as well. Another
advantage is the ability to incrementally update a system, or parts of a
system, from the command line. In fact, having all system administration
tasks be commandline and script-oriented is a real plus on a cluster, in my
The flexibility of Portage/Gentoo is one of the reasons for using it on a
cluster. My current setup has the Portage tree being shared over NFS to the
nodes, with squid on the master providing a proxy for emerge. I also have
some homebrew scripts for cloning using rsync. I have other configurations in
mind to try, like installing the node OS in a chroot environment on the
master, and just propagating it from there to the nodes.
Making Xfce the desktop manager on the nodes has also been a big improvement
over using Gnome (on Fedora), which was just massive overkill.
One thing I think Gentoo needs is its own version of the SystemImager tool. A
'GentooImager' ebuild, if you will. SystemImager is tailored to
Redhat/Fedora, and includes the whole kitchen sink in its distribution. A
GentooImager ebuild, perhaps based on and borrowing from SystemImager, could
just make the necessary tools -- dhcp, syslinux, tftp, etc. -- be dependencies.
Beyond that, there's the matter of providing an environment for pxeboot to
boot into and bootstrap a system and the other details of making system
images. Anyway, its something to think about. When/if we build another
hyperwall, I may start working on such a package. We'll see. I'm interested
in hearing from others who have installed and maintained Gentoo on a cluster.
As for disadvantages, I don't see any particular disadvantages to using Gentoo
on the hyperwall, though it might be difficult coming at it as your first
Gentoo install. Areas of Gentoo that I see as needing improvement are in
handling security-specific updates, and in tracking important messages that
are printed during an emerge. I'm glad that these are issues that appear to
be getting attention.
Are there any future plans for Gentoo installations in your department?
Right now, I'm just the annoying Gentoo flag-waver of my small group. I've
managed to generate some interest among a portion of my group, and I think one
of them is going to give Gentoo a try. We'll see what happens.
As I mentioned, Gentoo plays a major role in the security department. As for
Columbia itself, well, one can dream...
How can you be contacted?
Contact information can be found via my web page
Heard in the community
contents of /dev after initial installation
The new dynamic device managers (deprecated devfs and udev) handle all
device nodes dynamically. So why are there lots of static device nodes
in /dev? Gentoo can boot without them, but for system rescue it is a
lot more convenient to have a fallback. You could remove all but one or
two static device nodes and still boot, only debugging would be
difficult as you might not have terminal output ...
India: FOSS.IN conference with Gentoo participation
Last week India saw its largest free and open-source software
conference -- Foss.in/2005 held at the Bangalore
Palace. Gentoo was officially represented by developers Seemant Kulleen and Shyam Mani, in addition to local
enthusiasts Ashish V, Ananth, and Arun Raghavan.
Figure 4.1: The FOSS.IN booth staff
The first appearance of Gentoo at the event occured in the form
of all the conference's servers and majority of the desktops running
Gentoo. The Gentoo stall that was set up in the expo area was rarely
without a crowd of both Gentoo and non-Gentoo users. Much interest was
shown in how Gentoo is different from other distributions and existing
users were assisted with troubleshooting their issues. More than 100 CDs
(installation and package) were burnt over a period of 2 days.
The afternoon of Day 2 saw an entire 250-seater hall dedicated to
everything Gentoo. Seemant kicked the session off with an introduction
to the history of Gentoo, and a brief run-down of what makes Gentoo
tick. Amidst much applause, Seemant handed the stage to Shyam who, after
dispelling the "Gentoo is for Ricers" line of thought, walked through
the various features of Gentoo that users would need to be aware of.
Arun followed this up with a quick Ebuild 101 session. Seemant returned
to present a number of case-studies of Gentoo in production
environments. Each talk ended with a number of questions from the
audience. The session was ended with an open-ended QA session and a
quick demonstration of how to write an ebuild. Though there were many
requests to demonstrate an actual installation, users were encouraged to
try this themselves, then seek assistance at the Gentoo stall if required.
In a short session with the localisation folks, an rough action plan to
include localisation and Indic font support was drawn up.There was also
talk of setting up a local Gentoo users group. Watch out for this in
The team also wishes to thank Harikrishnan for coming up with some
awesome posters that were made use of in the Gentoo Stall.
Germany: "Friends of Gentoo e.V." at the "Linuxtage in Essen"
Last weekend the German not-for-profit organization Friends of Gentoo e.V. managed
to set up a booth at the first Linuxtage in Essen. Besides the usual things like
demonstrating Gentoo on several notebooks and more obscure hardware (this time an
old Compaq desktop machine) the booth team sold a special and limited printed
edition of the German Gentoo Handbook. The first day of this new exhibition on the
open-source conference trail exceeded expectations and was a huge success, with more
than 1300 visitors. The second day, Sunday, was topped by a presentation about the
Gentoo Metadistribution, by Gentoo developer Tobias
Gentoo in the press
Retro Gaming Hacks (October 2005)
A new book from O'Reilly, Retro Gaming
Hacks, is dedicated to helping 20- and 30-something gamers get their
nostalgia on (while introducing younger gamers to those gems of our gaming
past). In addition to actually running retro games on real retro hardware,
the book gives readers the know-how to emulate classic gaming systems on
modern computers. Yes, even modern computers running UNIX operating systems.
And that, of course, is where Gentoo enters the picture. Hacks #22 (Play
Arcade Games Under Linux), #40 (Run NES Emulators Under Linux, #43 (Emulate
Other Class Systems in Linux), #51 (Use Console Controllers on Your PC), and
#52 (Use USB Gamepads Under Linux) were written by Gentoo developer Josh
Glover, who makes sure to point out how easy it is to install emulators in
Gentoo Linux as opposed to other distros. Retro Gaming Hacks should be
available from finer booksellers everywhere; or read it online using
O'Reilly's Safari Bookshelf.
Gentoo developer moves
The following developers recently left the Gentoo project:
The following developers recently joined the Gentoo project:
- Alexandre Buisse (Nattfodd) - text-markup
The following developers recently changed roles within the
Inkscape: Buffer overflow
A vulnerability has been identified that allows a specially crafted SVG
file to exploit a buffer overflow and potentially execute arbitrary code
For more information, please see the GLSA Announcement
chmlib, KchmViewer: Stack-based buffer overflow
chmlib and KchmViewer contain a buffer overflow vulnerability which may
lead to the execution of arbitrary code.
For more information, please see the GLSA Announcement
The Gentoo community uses Bugzilla (bugs.gentoo.org) to record and track
bugs, notifications, suggestions and other interactions with the
development team. Between 28 November 2005
and 05 December 2005, activity on the site has resulted in:
- 717 new bugs during this period
- 310 bugs closed or resolved during this period
- 28 previously closed bugs were reopened this period
Of the 9129 currently open bugs: 97 are labeled 'blocker', 207 are labeled 'critical', and 560 are labeled 'major'.
Closed bug rankings
The developers and teams who have closed the most bugs during this period are:
New bug rankings
The developers and teams who have been assigned the most new bugs during this period are:
Please send us your feedback and
help make the GWN better.
GWN subscription information
To subscribe to the Gentoo Weekly Newsletter, send a blank email to
To unsubscribe to the Gentoo Weekly Newsletter, send a blank email to
from the email address you are subscribed under.
The Gentoo Weekly Newsletter is also available in the following