Gentoo Monthly Newsletter Howto

1.  Overview

So, you want to be a GMN editor, eh? This guide will show you how to write a Gentoo Monthly Newsletter from start to finish. It's more or less structured like the newsletter itself.

It's very important to familiarize yourself with GuideXML, the language the GMN's written in. Pay careful attention to coding style; you want the code to be nice and easy to read, both for yourself, and for your teammates. So take the time to read past issues, and especially view the source code! You can view the source online by appending ?passthru=1 to the .xml URL of any newsletter. Or just open up a copy from your local CVS checkout in your editor of choice.

Files

Putting together a newsletter requires several scripts and files. These can be found in our code repository. Be sure to save the following files to your GMN working directory:

• gmn_bugzie.py
• gmn_bugzie_aggregate.py
• get_packages_url.py
• print-dev-stats.py
• gmn2text.xsl
• pygooglechart.py
• gwn_adds_removes.py

There's also a skeleton newsletter available. This template will let you get a quick start on each monthly newsletter. Adjust the dates, times, locations, links, and numbers for each new edition. It's pretty straightforward.

2.  Articles

Feature articles are arguably the heart of the newsletter. However, articles don't usually just fall from the sky: you may have to actively scour the internet looking for articles, news releases, blogs, etc. pertaining to Gentoo. However, rather than do all this work yourself, you can make better use of your time by actively soliciting help from the user community and your fellow developers. Get them to send in articles and links of interest. The more they write up ahead of time, the better, as you'll need to do less editing. When someone sends interesting material your way, be sure to give them an author or contributor credit in the GMN. Participation should be fun and rewarding.

Gentoo news

Lead off the GMN (after the usual introduction) with general distribution news. The GMN is the first place people look for things like Gentoo release announcements, Council and Trustees meetings, and other important news. Oftentimes critical systemwide changes or security notes may be found here, such as the mask & removal of PHP4, stabilization of the latest Portage version, baselayout changes, and similar. Or mention projects that Gentoo is participating in, such as Google Summer of Code; see the April 2008 newsletter for a nice example.

The Council and Trustees post meeting summaries, minutes, and agenda both to the mailing lists and to their individual project pages. Use the summaries (with links to the full documents) here; this way you don't waste time duplicating their efforts. Occasionally there will be an important project meeting for security, desktop, Portage, etc. Make sure to include such meeting notes. Pester the project secretaries/leads for minutes and summaries if you have to. Remember, you don't have much time to play investigative journalist. Try to get the other projects to cooperate with you. Publicity should good for them, after all.

Next: the mini-calendar with the next month or three's events. Upcoming events might include monthly bugdays, Gentoo IRC meetings, and Linux/FOSS trade shows.

Gentoo international

The Gentoo International chapter reports the happenings at events around the world in which Gentoo and its developers have a presence. Pictures are always good!

In the past, the GMN has extensively covered events such as FOSDEM, SCALE, FliSol, LinuxTag, and various development summits in which Gentoo has a presence.

There won't always be a Gentoo International chapter to include, as events happen just a few times a year. But when they do, make sure to cover them! Talk to the developers and teams who went; check their blogs, etc.

Community news

The community news chapter can consist of:

• Interviews with individuals, companies, and other folks who use Gentoo for work, play, school, etc. This is one of the best ways to highlighting the things Gentoo is capable of, and how it's being used. Google Summer of Code interviews are another favorite for this chapter.
• Articles focusing on a particular bit of community-produced Gentoo-related software, like Himerge. Interest pieces on things that affect the community of developers and users alike, such as Sunrise or Bugzilla, may be found here. However, the GMN doesn't really cover things like individual non-Gentoo-specific applications, or applications that are not in the tree. "Community news" shouldn't be a general advertising space.
• Planet Gentoo, an aggregator of Gentoo developer blogs. Pick the best of the Gentoo-related posts from the past month and include one or two sentence summaries.
• "Gentoo in the News" is a good place to share sightings of Gentoo. Reviews, enterprise Gentoo usage, magazine appearances, new distributions based on Gentoo, and other Gentoo links & references go here. The more the merrier, even if they're just short tidbits.

Tips and tricks

This chapter contains useful information for administering your system, keeping things up-to-date, tweaking applications, monitoring boxes, and so on. If you're lucky, you can get users and other developers to email some collected tips or "best practices" each month. You may want to cull the forums for useful material.

3.  Statistics collection

The first half of the newsletter features articles, and those vary quite a bit month-to-month. The second half contains general statistics on Bugzilla, the Portage tree, developer changes, and so on. It's very formulaic, straightforward, but also can be the most time consuming part of the newsletter to write. This is where you'll use all the scripts we have to round out the GMN.

The statistical chapters are assembled as follows:

Bugzilla statistics

To generate the nice table of Bugzilla statistics (new bugs, closed bugs, distribution, etc.), do the following:

$ touch breport
$ python gmn_bugzie.py > breport
$ python gmn_bugzie_aggregate.py > gmn-bugzilla.txt (For safekeeping)

The first script, gmn_bugzie.py, generates a statistics file called breport after querying Bugzilla.

The second script, gmn_bugzie_aggregate.py, creates and fills in the statistics table that you copy into the GMN. It also generates three graphs: activity.png, opened.png, and closed.png. These three files come courtesy Google Chart (via pygooglechart.py). Save the graphs to your working directory; you'll be adding them to CVS later on.

Once you've got your statistics output saved to gmn-bugzilla.txt (or whatever you named it), you can paste it into the appropriate GMN section. Make sure to properly link in the generated graphs. Take a look at the code for the past issues to see how it goes.

Portage statistics

To create the Portage statistics (number of packages by keyword, distribution, etc.), you'll need an unmodified Portage installation. This means you shouldn't be using PORTDIR_OVERLAY or anything else that affects the number of categories and packages reported. Also, you'll have to use the downloaded metadata cache obtained from syncing, so you can't set various fun things like portdbapi.auxdbmodule = cache.metadata_overlay.database in /etc/portage/modules.

First, you'll need to emerge portage-utils, and run one of its utilities as shown.

$ qcache -s

Save the output to a file, as you'll need it in just a bit. Before you can create the Portage statistics graph, you'll need to manually edit get_packages_url.py. Fill in the values in the arrays declared on the first two lines to correspond with the values generated by qcache -s. Yes, it's a pain to manually edit the script every time you want to use it, but it'll do for now. (If you have a better way, be sure to email us!)

Once you've edited the script, run it:

$ python get_packages_url.py

This script creates the URL to a Google chart. Download the chart and save it to your working directory as keywords.png. You'll be committing this file with the rest of the charts soon. Again, make sure to add the proper link to this file within the GMN.

The package adds/removes are generated by saving the logs emailed by infrastructure, and running gwn_adds_removes.py on them:

$ python gwn_adds_removes.py add-removals.1220227200.log > foo.txt

In this example, the output of the command is saved to a file, rather than printed straight to the terminal. gwn_adds_removes.py will create the GuideXML; all you have to do is paste it into the newsletter.

Developer statistics

You can get the developer statistics (total recruited, away, etc.) by first downloading the devaway XML file, and then by hand-editing the print-dev-stats.py script to point at your own CVS checkout directory.

$ wget http://www.gentoo.org/dyn/devaway/devaway.xml?passthru=1 -O devaway.xml
$ python print-dev-stats.py

Once you've done this, paste the numbers into the developers summary section (number recruited, active, and away).

For developer changes (joining/leaving projects or teams), run a diff of the changes to herds.xml since the last issue. This information is available in CVS.

You should also keep a close eye on new developer emails sent to the gentoo-dev-announce and/or gentoo-dev mailing lists. You should receive automated retirement notices generated by Infra sent directly to the GMN email alias.

4.  Finishing up

As a courtesy to your fellow Gentoo developers, solicit feedback on the latest issue by writing to the gentoo-core mailing list at least one day in advance of publication. Be sure to provide them with a working URL to the issue; your devspace on dev.gentoo.org is good enough, as dev.gentoo.org automatically renders GuideXML into HTML.

Final touches

Once you've applied the final fixes from gentoo-core, you'll need to add a few more bits to the issue immediately before committing. Time is important here, as the webnodes take awhile to update.

First, make sure the dates inside the GMN issue are set to the correct date of publication. You'll also need to adjust the submission deadline date for the next issue; this information is found at the very end of the newsletter. You should also verify that the volume and issue numbers are correct; these are scattered through the first part of the GMN.

Next, post an announcement topic to the Gentoo Forums. Let the users know a new issue has been published; be sure to include a link (even though the newsletter isn't there yet. This forum thread is a convenient way for users to discuss the issue and provide feedback. Refer to the announcements for the August 2008 and July 2008 issues to see how it's done.

As soon as you post the announcement, add the link to the discussion thread to the issue. The forum link appears at the very beginning and very end of the issue, so make sure you fix both places.

Next, run gmn2txt.xsl. This script will create a plain-text version of the newsletter suitable for emailing to the GMN subscribers. See the usage notes inside the script itself if you need help.

Committing

Now go into your CVS checkout. You'll be committing several things all at once, so stay sharp. First, update the GMN index page at gentoo/xml/htdocs/news/en/gmn/ with a link to the latest issue.

Second, write up the front page announcement for www.gentoo.org and add it to gentoo/xml/htdocs/news/. See the August 2008 news item for how it's done. Add it to CVS (cvs add).

Third, cvs add the completed issue itself to gentoo/xml/htdocs/news/en/gmn/, and the images (including all those graphs you generated) to gentoo/xml/images/<issue-date>/.

Finally, cvs commit the whole thing in one go. Back up to gentoo/xml/ and make an atomic commit from this top-level directory. This ensures that the front-page announcement, issue + accompanying images, and the GMN index all get updated at the same time. Hopefully not too much time has passed between creating the forum announcement and actually releasing the GMN.

Now that the online edition of the GMN is published, you can send out the plain-text email version. Log in to dev.gentoo.org and upload the text version of the GMN so you can email it. Once logged in, run:

$ mutt -s "Gentoo Monthly Newsletter: XX XXXX 200X" -i 200XXXXX-newsletter.txt gentoo-gmn@gentoo.org
(For example: "31 August 2008" and "20080831-newsletter.txt")

Confirm the address, subject, and content. (If your default editor is vim, you'll need to type :wq to confirm.) Once in the mutt message screen, press Ctrl-T to change the charset of the content to utf-8. This is important, because by default the charset is unknown-8bit. Press y to send the newsletter once you've confirmed the charset change. A moderator of the list will confirm and the message will be sent.

There . . . the front page and forum announcements are in place, the newsletter is online, the email edition is sent . . . now you can take a break! At least until the emails for the next issue arrive. And it all starts over again . . .

5.  Resources

• GMN skeleton newsletter: use this template for each monthly issue. You'll find that pretty much all possible sections have been added; if you don't have a certain section, just delete it. Be sure to fill in appropriate dates, issue numbers, and links where marked.
• GMN project page
• Index of GMN issues
• GuideXML Guide
• Gentoo Forums: the Gentoo Chat subforum contains discussion and feedback threads for the GMN; you may want to search the archives.

The contents of this document are licensed under the Creative Commons - Attribution / Share Alike license.