Gentoo Linux Cascading/Stackable ProfilesHistorically, one of the nice things about Gentoo has been the lack of bloat. Dan Armak's eclasses are a perfect example of shared code being put into a separate file that can be read by all who need it. In a similar vein, the ${PORTDIR}/profiles directory hierarchy in Gentoo has been the victim of much bloating up lately. Cascading Profiles (also known as stackable profiles) take a similar "object-oriented" approach to the Gentoo system profiles.
So what do we gain from cascading the profiles? A cursory glance at each profile (take the 2004.3 profiles for each of x86, ppc, sparc and sparc64 as an example) reveals that there are a LOT of commonalities. The most obvious example is the virtuals file. Up until now, every time a new virtual was introduced, the virtuals files in each profile directory had to be edited to add that virtual to the profile. Also, for a basic Gentoo (GNU/Linux) system, there is a common set of packages that describes it. Why repeat this information for each profile we have (and each new profile that comes along)? Introduce cascading profiles. Chris PeBenito made the first jump into cascading profile by distilling out the basic set of packages that any *nix based system will require. ${PORTDIR}/profiles/base contains this description of a basic system. Every Gentoo profile thus far has had each of the packages outlined in the packages file in that directory. Additionally, each profile has the same virtuals as defined in the virtuals file there. Finally, those USE flags which are fairly architecture independent are described in the use.defaults file -- these USE flags are common to all the architectures thus far. Looking at the packages file, you might wonder where util-linux package went. Well, seeing as there are efforts to expand Gentoo into the worlds of GNU/Hurd, OpenBSD and so on, the thinking was that the base profile should be applicable to those as well. Admittedly, coreutils seems a little GNU specific, but one step at a time. Hopefully, once you read the files in the base directory, you will agree that for the most part, it describes an implementation agnostic minimal *nix system. The Default (Linux/GNU) Profile This profile describes in relatively high-level terms the so-called default-linux profile. Historically, every new architecture supported by Gentoo has had a default profile, and this is a high level replacement of that. Between the base profile and the default-linux profile, we should have a fairly complete description of a minimal GNU/Linux system. This is done by adding to (or removing from) the aggregate packages list, virtuals mappings and further defining the USE flag mappings (additional USE flags may be introduced or masked out). The Architecture Specific Profiles These profiles extend the default-linux aggregate profile by adding to the packages list (or removing from it), and also by redefining specific virtuals mappings and USE flags. Additionally, USE flags may be further masked here. This specificity is completely optional. In the current implementation, the sparc profile is further refined into a family of profiles for 32-bit SPARC architectures and 64-bit SPARC architectures, since the SPARC family has a lot of consistency and similarity. Additionally the MIPS sub-architectures will define sub-architecture profiles. These profiles essentially delineate, again in fairly high-level terms, the differences between the sub-architectures. As an example, gcc-sparc64 is required on the SPARC64 profiles, but not permitted on the SPARC32 profiles, yet they both use the same bootloader. Thus, the bootloader would be in the parent (architecture specific) profile, and the gcc-sparc64 package would be defined in the sparc64 sub-architecture profile. This is it: the nuts and bolts of the final profile being defined. This profile is used primarily to lock down specific versions (though the higher level profiles are also free to do so) of packages and define exactly what, for example, the sparc64-2004.0 profile is all about and how it differs from the sparc64-1.4 and sparc64-gcc33 profiles, or how the x86/2004.0 profile differs from the x86/gcc2 profile. Historically, every new architecture that Gentoo supports gets a default-${arch}-${version} directory in ${PORTDIR}/profiles. Invariably, they have been linux (with GNU userland) based profiles. The future, however, is open. So with cascading profiles, we can define different classes of profiles that extend the base profile. The first class, which all the current architectures will fall into, will be default-linux. The other profile classes, incidentally, are selinux and hardened. This document focuses exclusively on the default-linux profile class, because the other two are very specialised classes handled by the Hardened team. Now, the default-linux directory itself contains the files parent, packages, use.default, use.mask, and virtuals, along with directories which we'll cover in the next section. Let us begin with the parent file. This file points to the profile whose attributes we are inheriting. It contains a relative path to that profile. So, in this case, it would contain: ../base. The packages file simply extends the base profile's packages file. It does this in three ways:
The use.default file acts similarly. You can use this file to add further USE flag mappings for this profile class, or override the USE flag mappings described in the parent profile. The use.mask file is used to invalidate certain USE flags in this profile class. For example, the selinux USE flag is valid only in the hardened and selinux profile classes, but will wreak havoc on the default-linux profile class. So, it is masked here. Finally, the virtuals file extends the parent's virtuals file. You can again use it to add further virtuals mappings that would be valid in only this profile, or override the parent's mappings. With the architecture profiles, we are one step closer to completing the cascading profile. This is the level where new architectures will be added. So to recap, we have the base profile and the default-linux profile defining a basic minimal and architecture independent system. For most architectures, this level of the cascade (or stack, if you prefer) defines some specifics. For instance, the virtual/bootloader mapping can be defined at this level, since each architecture class uses a different default bootlader from the others. Additionally, there may architecture specific things that are required as part of the base system, such as: sparc-utils for SPARC. Some architectures prefer to mask out the architecture USE flags for the other architectures. For example. the sparc profile puts x86, ppc, alpha and mips into the use.mask at this level. This is an optional level in the cascade. The premise here is that if a family of architectures (for example, SPARC and MIPS) have basically the same requirements for a basic system with only a few differences (be they packages, virtuals or USE flags), those are expressed here. Part of the reason for the ballooning of profiles has been the addition of a profile for every new LiveCD release. When switching over your old profiles to the new cascading system, please analyse closely the real differences between your architecture's different profiles. In the case of moving the x86 profiles, we found that 1.4 and 2004.0 were identical. So, in the new system, they have been merged into the cascaded 2004.0 profile. Together, the releng and dev-portage teams determined that the ideal solution is to only create a new profile when absolutely necessary. The necessity is left up to the team leads and architecture releng people, but a good rule of thumb is that when there are major changes (for instance the gcc2 to gcc3 switch), a new profile is warranted. Additionally, it might be good to create an experimental profile to test out higher toolchain versions, and then later either deprecate the experimental profile, or migrate the changes over to the stable profile, leaving the experimental profile to move onward. Certainly, a discussion on this philosophy should be carried out, so what is written here has not been etched in stone. The final profile will contain one additional file indicating that it is, indeed, the leaf of the cascading tree. This file is the make.defaults file. Again, as above, the parent file will point to the parent directory. And, of course, the packages, use.defaults, use.mask and virtuals files can be overridden and extended. Ideally, here is where specific versions of things can be locked down (or at the subarchitecture level, if there is one). Use this to concretely define your profile. Admittedly, there's a bit of tedium involved in migrating your existing profile to the cascaded scheme, but the payoff at the end is invaluable. For instance, new virtuals need no longer be added to every single virtuals file in every single profile directory. Once they're added just once at the base or default-linux level, everyone can enjoy the utility of it. If there are any questions, please direct them to Seemant, ZHeN or any member of the releng team. The contents of this document are licensed under the Creative Commons - Attribution / Share Alike license. |
|
