Gentoo Monthly Newsletter: 24 April 2008

1.  Introduction

This month in the GMN

Welcome to the April issue of the Gentoo monthly newsletter! This month, we haven't made any significant changes from the previous edition. However, we have featured an interview, and we hope to include more of them in future issues. You'll note that we will be interviewing not only Gentoo developers, but also people involved in the Gentoo community at large.

We also had to delay the publication of this issue by a few days in order to accommodate the announcement of selected projects for this year's Summer of Code.

As usual, you can discuss any aspect of this issue of the GMN in the corresponding forum thread. We look forward to hearing from you!

2.  Gentoo News

baselayout-2 and OpenRC hit the tree

After a long wait, baselayout-2 and OpenRC have finally hit the unstable tree. For those of you who want to be on the bleeding edge, please make sure you read the migration guide before attempting to upgrade. Both packages are in ~arch for a reason. We recommend you don't upgrade these packages on machines where stability is of prime importance.

Summer of Code 2008

The list of selected students for the Google Summer of Code 2008 was announced today. Gentoo is happy to mentor 6 students on proposals which were ranked highest by our developers:

• Using Gentoo, Seed Linux and Catalyst, provide an easy access to a Beowulf clustering/HPC environment to everyday users: by Eric Thibodeau. Mentored by Donnie Berkholz.
• "Automate It All": by Nirbheek Chauhan. Mentored by Stephen Klimaszewski.
• Implement OpenPAM compatible modules for Linux: by Serafeim Mellos. Mentored by Diego Pettenò.
• GNAP Love (embedded framework enhancements): by Vít Vomáčko. Mentored by Andrey Falko.
• "Setting Beacon Afloat": by Nandeep Mali. Mentored by Anant Narayanan.
• revdep-rebuild reimplementation: by Carl Lucian Poston. Mentored by Marius Mauch.

We will be featuring interviews with all our students in the coming months. All the best to our students!

Gentoo is always looking for new contributors. You can help us in a variety of ways, not all of which are technical. Check out our guide to becoming a new developer for more information.

Council Meeting Summary

The Gentoo Council held its monthly meeting on April 10, 2008. The items put up for discussion were:

• Minimal activity for ebuild devs: Current activity required is 1 commit every 60 days. Should it be higher? Agreement was hard to find. Some people thought it should be 1 commit / week, others said that people have busy lives and questioned the benefits. A number of people did agree that we should trust the judgment of the undertakers, and that the number of commits is not the only activity indicator.
• Initial comments on PMS: Are there any major changes needed, or just tuning details? The council voted that kdebuild-1 and other unapproved EAPIs could not be in an approved PMS document. The spec isn't a place for proposals or things that will never be submitted for approval by the council. It's a specification, a reference of what is allowed in the main tree.

Some items were rolled over from the previous meeting:

• GLEP 46 (Allow upstream tags in metadata.xml): Approved.
• Document of being an active developer: No updates.

Coming Up

• Bugday: Looking for a way to help out Gentoo without investing a lot of time? Join us on May 03 for our monthly bugday, and help us squash some bugs.
• Trustees Meeting: There will be a single topic trustees meeting on May 04 - #gentoo-trustees on irc.freenode.net at 1900 UTC - with the aim of adopting the bylaws. All are welcome to attend.
• Council Meeting: The Gentoo Council meets every month to discuss important technical issues that affect Gentoo as a whole. This month's meeting is scheduled to be held on May 08, and everyone is welcome to participate - #gentoo-council on irc.freenode.net at 2000UTC.
• LinuxTag Berlin: The biggest European Linux event is on again. On the Berlin Fairgrounds, Gentoo will be featured again. Join developers and users on the booth between May 28 and 31, we hope it'll be as much fun as Chemnitz was!

3.  Heard in the Community

Interview: Gentoo in a production environment

This month, we had the opportunity to catch up with Ramon van Alteren, team leader of the system engineering department at Hyves. Hyves is a European social networking site akin to MySpace, FaceBook or Orkut.

GMN: Please give us a brief introduction of yourself and your role at Hyves.

Ramon: I'm a long time Gentoo user both personally and professionally and started using Gentoo somewhere in 2002 coming from Mandrake. Before that I'd been using Linux for about 5 years.

Hyves is a social network site akin to MySpace, FaceBook, Orkut etc. The website was originally started by Koen, Floris and Raymond, I joined the company in November 2005 as the first paid sysadmin to help with managing the 40+ Gentoo linux servers.

The System Engineering department currently is a team of 9 people who manage all servers from bare-metal provisioning up to the application software including the network infrastructure. The website itself is created by the developers (a team of 14 people).

GMN: What is the scale of your website?

Ramon: We currently have over 6 million registered users and counting, peak performance is currently at 150+ million page views per day (13 million per hour). These are filtered page views, raw http requests on our web farm are well over 20.000 req/sec.

On our mysql database backend, we serve roughly 100.000 actions/sec at peak performance. Aside from the web frontend and the database backend we have a very large file-storage for media that our users upload. The total used storage totals around roughly about 280 Terabytes. All in all with all hosts accounted for we have 1800 servers most of them with 2 or 4 cores each. All of these run Gentoo Linux. :-)

GMN: Why did you choose Gentoo Linux to power your infrastructure? What features in particular made you choose Gentoo over all the other distributions out there?

Ramon: We've been running on Gentoo Linux since the inception of the website. There has been a very short period at the very beginning when the company owned two colocated servers which came with Fedora but that was a very long time ago. Our main reasons to switch to Gentoo were:

• USE flags (the ability to easily switch additional functionality on or off)
• Extreme customizability (custom ebuilds, install locations etc.)
• Close tracking of upstream (speed of updates)
• Good community support
• Excellent documentation

We're using some very bleeding edge software in some cases to run our website with fairly extensive customizations in general to adjust for scale and/or performance. Portage, overlays and the ebuild system in general go a long way to achieve that while staying within the Gentoo distribution framework for all the other stuff.

The fact that a bare-bones Gentoo install doesn't come with all kinds of cruft and distribution-centric customizations, helps to keep our systems mean and lean.

In upgrading from php-4 to php-5 we've found slotting to be an invaluable feature which eased the migration a great deal. Additionally we had to stay on php-4 for some time after Gentoo deprecated support for it, pulling in security patches with an ebuild turned out to be fairly simple, enabling us to migrate at our own pace.

GMN: Please describe how you used the tools Gentoo offers to make maintaining such a huge pool of servers easier.

Ramon: Most of our system administration is not done using Gentoo tools but by using an automated configuration management system for Unix hosts called puppet. It uses eix to build a package database.

Among the tools Gentoo offers:

• We use catalyst to build our own stage3 and stage4 tarballs.
• We use quickinstall by Andrew Gaffney as part of our provisioning framework
• Catalyst relies on genkernel to build our kernel
• We use Portage's binary package support for critical fixes

GMN: Have you been through any major "stumbling blocks" while setting up your infrastructure? If so, how did you go about resolving them? Did you find the various support avenues that Gentoo offers to be helpful at the time?

Ramon: Several of course! Most of them had absolutely nothing to do with Gentoo but more with bugs and performance issues in various software packages.

With respect to community support, we actively search and query IRC, bugzilla and sometimes the forums as well. Additionally we track the gentoo-dev list for changes. The forums tend to be a valuable source of information for desktop issues we might have had (most of the system engineering team runs Gentoo as desktop OS). However for large-scale server issues it has a bit too high noise/content ratio because of its desktop oriented nature.

We try to give stuff that we learned back to the community by filing bugs, if possible with patches. However we find that most of the stuff we customize is of fairly limited interest to the larger community because of its focus on our own environment.

Running a server park this large is usually eats away most of the time available, so there is little left to clean up patches and/or ebuilds to make them more generally usable. We do try to file amd64 keyword stabling reports on stuff we deploy in our server park that hasn't been marked stable (yet).

GMN: Is there a feature that you wish Gentoo had?

Ramon: Let's see [rumbles around desk for the long list of features we'd wish gentoo had]. More seriously, the list is actually surprisingly short:

• Currently I'm working on getting stackless python working in an agreeable fashion with the rest of the python framework in Gentoo. We've rolled out a preliminary stackless ebuild which basically installs it side by side with the normal python setup in /opt but I consider this to be sub-optimal at the least. We haven't finished it yet, but I think it will be doable.
• I'd really really like to see a release-based Portage snapshot released together with the actual release. However, that is not of much use without a snapshot from the distfiles mirrors at the same time. This would make Gentoo less of a moving target and a more reliable base to build servers on. There has been talk about this on several mailing lists where people invariably object that this would create a false sense of stability because no one would port security patches and/or issues but as far as we are concerned that would not be necessary. I understand the constraints placed on the mirror system by such a setup. We are doing this internally for our stage4 files at the moment and that's pretty doable, but then, we only mirror the distfiles part (that we actually use in our stage4 builds) and not the entire Gentoo release with all the related desktop software.
• We depend fairly heavily on the binary package support in Gentoo, and we need to separate packages over multiple ServerOS images (stage4 builds) if we need them with different USE flags. I would like proper USE flag support for binary packages.
• A little less aggressive pruning of ebuilds in the Portage tree would help. A lot of people are probably still running apache-2.0.x versions, as we are. They've left the Portage tree which is kind of sad, I miss them!

GMN: Any concluding comments?

Ramon: The beauty of Gentoo is that it offers a flexible framework for building a highly customizable linux base to run your application on without getting in the way. That flexibility and customizability brings its own complexity but as far as I'm concerned it has been worth the trouble.

Running this kind of infrastructure with just 9 people is a form of highly organized madness! If there are any developers or sysadmins out there with a healthy interest in high-performance large-scale infrastructures and Amsterdam I'd like to talk to them. We have open positions in both development and system engineering and would prefer to hire people from the Gentoo community.

We welcome feedback from the community, you can reach me personally by email or poke me on IRC (My nick is Innocenti on the Freenode network). If you're interested in working with us, please don't contact me directly but send mail to: jobs.sysadmin@hyves.nl.

And last but not least I'm really pleased to make the following announcement:

Based on a recent discussion with Robin H. Johnson and Alex Howells from gentoo-infra, Hyves will sponsor the Gentoo community by helping out with new servers for Bugzilla. We are putting up two large AMD64, 16Gb servers with fast SCSI disks for the database backend and 2 beefy webservers to improve the current bugzilla situation. We're currently working with infra to get the stuff up and running and I hope the servers to go live soon.

GMN: Thank you, Ramon, for your time and for speaking with us!

Figure 3.1: The team at Hyves (L-R): Jeffrey Lensen, Marlon de Boer, Eugene Molenaar, Ramon van Alteren, Gerwin Scheeve, Maarten van der Bogaard, Stefan van der Wiele and Frank Zwart

Planet Gentoo

The good user: Ben de Groot explains to us in a series of posts, how one can become a successful Gentoo user and stay that way.

Baselayout status: Doug Goldstein informs us about the status of the new baselayout.

Translations: Translations are an important part and free software has spread into markets because of translation into some uncommon languages. Diego Pettenò shares his thoughts on ebuild internationalization.

Alternative package search engine: Steve Dibb implemented the old design of packages.gentoo.org with new code.

Workplaces: Ever wanted to know where Gentoo is hand-crafted with lots of love? Some developers setups are presented in a few blog posts.

Man pages with Vim: Ali Polatel tells us how to view man pages from your Vi-like editor.

Playstations and Gentoo: Although older Playstations are MIPS based, Stuart Longland tells us why they are not supported.

Book review: For our German readers, Christian Faulhammer posted a little review on the second German Gentoo book by Gunnar Wrobel.

Go for Gold: Maybe you have heard about the new linker in binutils. Bernard Cafarelli tells us how to use it beside a stable version for your own projects.

Gentoo in the News

PC Magazine: An article featuring the Zonbu PC mentions that it runs Gentoo Linux - "Though sold through a nontraditional channel there's no hard drive, so you pay monthly for online storage the Zonbu PC is a good alternative to the traditional desktop PC. It runs Gentoo Linux, which is a lot more usable than the gOS Linux on the gPC. This is the EPEAT Gold and Energy Star 4.0 certified PC to buy if your computing needs run mainly on the Web. It's the first PC Mag Green Approved product ever". For more information, check out the complete article.

Linux Devices: PC-Doctor, a well known software vendor has developed a new Network Factory appliance, which is currently in the prototype phase. The device runs Gentoo, on an undisclosed hardware platform. The company chose the Gentoo distro, "because we know it," said van Aman. "We chose a source-based distribution because we only need to support a few hardware configurations, and we want to squeeze the maximum performance out of the hardware that we have". For more information, check out the complete article.

ITWire: A news item on the website describes how a user used Gentoo to create his own distribution - Kororaa. Check out the complete article for more information.

4.  Tips and Tricks

Speedup Boot Time

Are you impatient when booting your computer? If yes, try the following:

$ $EDITOR /etc/conf.d/rc
(Change)
RC_PARALLEL_STARTUP="no"
(To)
RC_PARALLEL_STARTUP="yes"

(Assuming you are using GRUB and /boot is mounted correctly)
$ $EDITOR /boot/grub/grub.conf
(Append the 'quiet' option to the kernel you use, for example:)
kernel /boot/vmlinuz-2.6.23.9 root=/dev/hda2 quiet

(View what services are currently activated on boot)
$ rc-status boot
(Check what is good with a default runlevel)
$ rc-status default
(For example, you probably won't need keymap if you're building a carputer:)
$ rc-update del keymap

If you build your kernel with no modules, you can remove modules from boot.

You may also like to install BootChart, a tool for performance analysis and visualization of the Linux boot process.

IRSSI and BitlBee

Irssi and BitlBee are both great for simple chat clients. If you have a shell available, we recommend you run them off the server, since you will never logoff.

# emerge irssi bitlbee screenie

A previous GMN featured tips and tricks on screenie. Now, configure bitlbee to your taste, and start it:

$ /etc/init.d/bitlbeed start

Now start screenie and start an irssi session:

$ /connect 127.0.0.1

Now follow the instructions and connect into your IM accounts and IRC, never worrying about logging off.

5.  Gentoo developer moves

Summary

Gentoo is made up of 259 active developers, of which 44 are currently away. Gentoo has recruited a total of 637 developers since its inception.

Moves

The following developers recently left the Gentoo project:

• None during this period

Adds

The following developers recently joined the Gentoo project:

• Ahmed Ammar (b33fc0d3) - amd64
• Josh Glover (jmglov) - CJK

Changes

The following developers recently changed roles within the Gentoo project:

• Alfredo Tupone (tupone) joined the net-zope herd
• Ben de Groot (yngwin) joined the desktop-misc herd

6.  Portage

Summary

This section summarizes the current state of the Portage tree.

Figure 6.1: Package distribution by keyword

The following section lists packages that have either been moved or added to the tree. The package removals come from many locations, including the Treecleaners and various developers.

Removals:

Additions:

7.  Bugzilla

Statistics

The Gentoo community uses Bugzilla (bugs.gentoo.org) to record and track bugs, notifications, suggestions and other interactions with the development team. The following chart summarizes activity on Bugzilla between 16 March 2008 and 20 April 2008.

Figure 7.1: Bug activity split-up

Of the 10812 currently open bugs: 15 are labeled blocker, 106 are labeled critical, and 361 are labeled major.

Closed bug ranking

The developers and teams who have closed the most bugs during this period are as follows.

Figure 7.2: Bug closed rankings

Assigned bug ranking

The developers and teams who have been assigned the most bugs during this period are as follows.

Figure 7.3: Bugs assigned rankings

8.  Getting Involved

The GMN relies on volunteers and members of the community for content every month. If you are interested in writing for the GMN, do write in to gmn-writers@gentoo.org with your articles in plaintext or GuideXML format.

We solicit feedback from all our readers on the newsletter. If you have any ideas for articles, sections, or have anything to say about the GMN, don't hesitate to email us at gmn-feedback@gentoo.org.

You can also give us your feedback and comment on this particular issue of the GMN on the forum thread.

9.  GMN subscription information

To subscribe to the Gentoo Monthly Newsletter, send a blank e-mail to gentoo-gmn+subscribe@gentoo.org.

To unsubscribe to the Gentoo Monthly Newsletter, send a blank e-mail to gentoo-gmn+unsubscribe@gentoo.org from the e-mail address you are subscribed under.

10.  Other languages

The Gentoo Monthly Newsletter is available in the following languages:

• English
• German
• Italian
• Polish
• Spanish
• Simplified Chinese