Gentoo Overlays: Developers' Guide

1.  Introduction

Audience

This document has been written for Gentoo developers and users that want to contribute. If you just want to start downloading and using overlays, please see the Gentoo Overlays User Guide instead.

Who Can Use overlays.gentoo.org?

Any Gentoo project, Gentoo developer, or Gentoo user can have their own overlay hosted on *.overlays.gentoo.org, with the RSS feed from the changelog included on the overlays.gentoo.org planet.

Any User can download and use the contents of any hosted overlay. If you choose, you can also give users write access to your overlay.

What Does overlays.gentoo.org Give Me?

The *.overlays.gentoo.org service currently provides:

• Trac: a wiki with integrated subversion browser), for quickly creating and maintaining documentation about your Subversion based overlay
• Publishing the changelog for your overlay on the o.g.o homepage, so that everyone who's interested can see what's going on
• Gitweb: provides full-fledged web interface for viewing Git repositories.

... all hosted on secure, backed-up Gentoo infrastructure, administered by the Gentoo Infrastructure team (hardware / base OS) and the Gentoo Overlays team (wiki / VCS / ACLs).

Each overlay has separate authentication lists for Trac, Subversion and Git. It's no problem at all to give someone write access to just Trac (e.g. for writing documentation) without giving them write access to Subversion.

Why Should We Use overlays.gentoo.org?

You don't have to. You don't have to have an overlay at all, and if you do have one, you are absolutely free to host your own overlay somewhere else. You don't have to host an overlay on o.g.o for it to be considered "official".

The advantage of using overlays.gentoo.org is that we already have everything setup for you. You don't need to admin your own server, or worry about software upgrades. We take care of all of that for you.

When Should We Not Use overlays.gentoo.org?

The purpose of o.g.o is to help bridge the gap between developers and users. Gentoo is a community-based distribution, and we believe that our users are just as important a part of that community as developers are.

All of the overlays hosted on o.g.o are there for all users to download and use. It's for users to make their own decision about what software they install on their computers - and that includes choosing to use your overlay. Some users will make bad decisions, and end up breaking their computer. They may even end up blaming Gentoo as a result. That's okay; these people probably go around blaming everyone but themselves for their own mistakes anyway, and there's probably nothing you can do to change that. But that still doesn't give any of us the right to choose for them.

Users are free (in fact, they are encouraged) to provide constructive feedback about anything to do with Gentoo - including all of the overlays hosted on o.g.o. That feedback can come via bugs.g.o, via email to your project team or directly to you, via the forums, or via IRC. We're not talking about genuinely abusive users; we have no time for those, and no-one expects you to have any time for them either.

o.g.o used to have restrictions of not being the $UPSTREAM for packages. This restriction has been adjusted. We do offer hosting as the $UPSTREAM now, but only for packages that are Gentoo-specific or important to the running of Gentoo. Other hosting may be more suitable, some services in this vein are: SourceForge.net, Berlios, Patrick's GentooExperimental.org, GitHub or Gitorious.

2.  Requesting An Overlay

Introduction

There are three types of overlays: "project" overlays, "developer" overlays and user overlays. The only difference between them is responsibility and accountability.

Project Overlays

"Project" overlays are overlays for official Gentoo projects. An example is the PHP Overlay.

An official Gentoo project is a project that has a project page on www.gentoo.org, and that has an elected lead. (This definition comes from the metastructure document). The project lead(s) are responsible for the project overlay, including its contents, and any problems it causes other Gentoo projects and developers.

To request a SVN project overlay, the project's lead just needs to pop into #gentoo-overlays on IRC and ask for an overlay to be created. Or, if he/she prefers, drop an email to overlays@gentoo.org. We'll take care of the rest, including granting write access to all the members of your project (as listed on your project page).

To request a Git project overlay, just visit the git.overlays site, and follow the setup instructions, emailing the completed template as directed.

For an SVN request, we will:

• create your overlay (trac site + svn)
• add your overlay's RSS feed to the o.g.o homepage
• create an o.g.o account for you if you don't already have one
• give you write access to your overlay's Trac wiki and Subversion repository
• give write access to all project members who already have an o.g.o account

For an Git request, we will:

• create your overlay (git, gitweb, no trac)
• add your overlay's RSS feed to the o.g.o homepage
• create an git.o.g.o account for you if you don't already have one
• give you write access to your overlays Git repository
• give write access to all project members who already have an o.g.o account

Developer Overlays

"Developer" overlays are overlays owned by individual Gentoo developers. One example is beandog's overlay.

If you have an @gentoo.org email address, and you've passed the ebuild quiz, then you can have your own developer overlay on o.g.o.

To request a SVN developer overlay, just pop into #gentoo-overlays on IRC and ask for an overlay to be created for you. Or, if you prefer, drop an email to overlays@gentoo.org.

To request a Git developer overlay, just visit the git.overlays site, and follow the setup instructions, emailing the completed template as directed.

For an SVN request, we will:

• create your overlay (trac site + svn)
• add your overlay's RSS feed to the o.g.o homepage
• create an o.g.o account for you if you don't have one already
• give you write access to your overlay's Trac wiki and Subversion repository

For an Git request, we will:

• create your overlay (git, gitweb, no trac)
• add your overlay's RSS feed to the o.g.o homepage
• create an git.o.g.o account for you if you don't already have one
• give you write access to your overlays Git repository

A Word About Accounts

Because o.g.o is designed to support a blend of both Gentoo developers and Gentoo users, we don't create 'real' system-level accounts on the o.g.o host.

3.  Working With Your Overlay

Introduction

You can access your overlay as soon as it has been created. Project and developer overlays have different URLs, so that everyone can tell one from the other, but otherwise they are identical in every way.

There are *no* read restrictions on overlays or wikis. Everyone has full read access to all overlays and wikis. If you need a 'secret' overlay of some kind, then o.g.o is not for you.

Accessing Project Overlays

If your project overlay is called 'foo', your Trac wiki site will be here: http://overlays.gentoo.org/proj/foo/

To checkout your Subversion repository, use:

svn co https://overlays.gentoo.org/svn/proj/foo/

While you may perform read-only actions via unsecure HTTP, you must perform all commits via HTTPS. If you need to switch between modes, use:

svn sw --relocate http://overlays.gentoo.org/svn/proj/foo/
https://overlays.gentoo.org/svn/proj/foo/

We maintain a full list of project overlays hosted on overlays.gentoo.org.

Accessing Developer Overlays

If your Gentoo email address is 'foo@gentoo.org', your Trac wiki site will be here: http://overlays.gentoo.org/dev/foo/

To checkout your Subversion repository, use:

svn co
https://overlays.gentoo.org/svn/dev/foo/

While you may perform read-only actions via unsecure HTTP, you must perform all commits via HTTPS. If you need to switch between modes, use:

svn sw
--relocate http://overlays.gentoo.org/svn/dev/foo/ https://overlays.gentoo.org/svn/dev/foo/

We maintain a full list of developer overlays hosted on overlays.gentoo.org.

Getting Started With Trac

Your overlay comes with Trac. Trac is a wiki, a subversion repository browser, and a bug tracking system that's very popular with open source developers.

We have disabled the bug tracking system in Trac. Use Gentoo's bugzilla for bug tracking your overlay.

Your overlay's RSS feed - the one that is shown on the o.g.o homepage - comes from Trac's Timeline page or GitWeb's summary.

Getting Started With Subversion

The advantages of Subversion over CVS include real versioning of directories, full changeset support, and it's much easier to do branching if you need to. The main disadvantage of Subversion is that it is slower than CVS, and that a local Subversion checkout requires more disk space.

If you have never used Subversion before, the online book is an excellent way to learn Subversion. You can buy it in dead-tree format too if you prefer.

Here are some basic commands to get you started.

svn co https://overlays.gentoo.org/proj/php

svn status

svn add my.ebuild

svn commit -m 'My changelog entry'

Layman

We are telling users to use layman to download and manage your overlay. layman is a utility written by Gunnar Wrobel which makes it very easy for users to work with overlays.

To get started with using layman, see the homepage documentation, the Gentoo Weekly News for 22nd May or read man layman.

Auto-Sync From Portage

Your packages in the Portage tree are always at risk of being changed without you knowing about it in advance. Arch teams need to be able to keyword packages (and fix arch-specific brokenness), the QA team fix perceived standards violations, and occaisionally developers will edit packages that they shouldn't.

You need to make sure that the changes made in Portage aren't lost the next time you copy your packages from your overlay back into Portage.

The PHP team have solved this problem by automatically copying their packages from Portage back into a 'portage' branch of their overlay every night. They can then use Subversion (or Trac's timeline) to see what has changed every day.

4.  Using git on overlays

Initializing your overlay

Before uploading you need to locally create a git repository and add all items:

cd ~/my-overlay

git init
git add .
git commit -m "populate overlay"

Note that this commit was only locally, now we get the server into the game.

git remote add origin git+ssh://git@git.overlays.gentoo.org/(proj or dev)/(name)

git push origin master

Source: http://www.kernel.org/pub/software/scm/git/docs/tutorial.html

Checking out the overlay with git

git clone git://git@git.overlays.gentoo.org/(proj or dev)/(name)/

5.  Giving Access To Your Overlay To Others

Introduction

One of the key features of o.g.o is that people who do not have write access to the Gentoo Portage package tree can have write access to one or more overlays. Several Gentoo projects have found that this is a great way to train and evaluate potential Gentoo developers in a safe environment.

Project Overlays: Giving Write Access To Team Members

Any developer listed on a team's project page on www.g.o can have write access to the team's overlay. The project lead can ask on their behalf, or the developer can come and ask for access themselves.

If the developer doesn't have an o.g.o account yet, he/she will need to open a bug at Gentoo's bugzilla so that we can create an account for them.

Project Overlays: Giving Write Access To Other Gentoo Developers

Any Gentoo developer *not* listed on a team's project page on www.g.o can have write access to the team's overlay. The request for write access must come from a member of the team. It doesn't have to come from the project lead.

If the developer doesn't have an o.g.o account yet, he/she will need to open a bug at Gentoo's bugzilla so that we can create an account for them.

Project Overlays: Giving Write Access To Gentoo Users

Any Gentoo user can have write access to the team's overlay. The request for write access must come from one of the project's leads. You can request that we give the user write access to Trac, to Subversion, or to both. (We will assume that the request is for write access to both, unless you say otherwise).

We cannot accept these requests from anyone other than a project lead. If your project only has the one lead, we recommend electing a second lead. If your one and only lead is AWOL, consider electing a replacement :)

If the user doesn't have an o.g.o account yet, he/she will need to open a bug at Gentoo's bugzilla so that we can create an account for them.

Developer Overlays: Giving Write Access To Gentoo Developers

Any Gentoo developer can have write access to your developer overlay. The developer can ask us directly; we will not give access until we've checked with you. You can also ask us to give write access to any named developer.

If the developer doesn't have an o.g.o account yet, he/she will need to open a bug at Gentoo's bugzilla so that we can create an account for them.

Developer Overlays: Giving Write Access To Gentoo Users

Any Gentoo user can have write access to your developer's overlay. The request for write access must come from you. You can request that we give the user write access just to Trac, just to Subversion, or to both. (We will assume that the request is for write access to both unless you say otherwise).

We cannot accept these requests from anyone else other than you. If you find yourself giving access to a lot of other people, it might be that you should consider setting up a new project, and transfering your work there instead.

If the user doesn't have an o.g.o account yet, he/she will need to open a bug at Gentoo's bugzilla so that we can create an account for them.

6.  Accessing Someone Else's Overlay

Using An Overlay

Everyone has full read access to every overlay. We recommend that you use

  emerge layman
  echo 'source /var/lib/layman/layman.conf' >> /etc/make.conf

After that, to view the list of overlays, use

layman -L

To install an overlay, use

layman -a <overlay-name>

You can now install packages from the overlay.

Requesting Write Access

If you want write access to a project overlay, contact a member of the project team, and ask them for access. If they approve your request, they will arrange for you to have write access, by contacting the Overlays team.

If you want write access to a developer's overlay, contact the developer directly, and ask them for access. If they approve your request, they will arrange for you to have write access, by contacting the Overlays team.

7.  Frequently Asked Questions

o.g.o Administration

Q: How do I contact the o.g.o admin staff?

• A: You can pop into #gentoo-overlays on IRC, and talk to us there. The current staff are mostly in European timezones.
• A: You can send an email to overlays@gentoo.org. Someone will answer you as soon as possible.

Q: Why can't I edit the access control list directly?

• A: (SVN) The access control list currently lives in htpasswd format. Only o.g.o admin staff members have ssh access to the o.g.o box.
• A: (Git) The access control list currently lives in the gitosis-admin repository, which is writable only by the o.g.o staff.

Security

Q: Is my overlay available via https?

• A: Yes, it is.

Multiple Overlays

Q: Can I have multiple overlays?

• A: Yes, in a fashion. Inside your overlay, you can create sub-directories, and put separate package trees inside those sub-directories. Please take a look at the PHP project overlay for an example.

Importing Existing Overlays

Q: I already have an overlay, and I'd like to move it to o.g.o. How do I go about doing that?

• A: Create a tarball of your subversion repository, and put it somewhere where it can be downloaded via http. We'll download it and install it onto o.g.o for you.

Q: I have an overlay, but it doesn't use Subversion. How do I go about moving it to o.g.o?

• A: Ask us to create a new, empty overlay for you. You can then use 'svn import' to import your files into the new overlay. You'll lose your history, but that can't be helped.
• A: Search the Internet, and see if there is a tool to convert from your existing version control software to Subversion. If there is, use that, and then we can help you move it to o.g.o.
• A: If your version control software is used by Trac, and it can be used over HTTP, come and help us add support for your version control software on o.g.o.

"Official" Overlays

Q: When is an overlay considered "official"?

• A: An "official" overlay is an overlay managed by a Gentoo project (for project overlays) or a Gentoo developer (for developer overlays).

Q: Does an overlay have to be on o.g.o to be "official"?

• A: No.

The contents of this document, unless otherwise expressly stated, are licensed under the CC-BY-SA-2.5 license. The Gentoo Name and Logo Usage Guidelines apply.