Gentoo/AMD64 Testing Docs

Content:

• Gentoo/AMD64 Testing Docs
  Provides testing documentation for Gentoo/AMD64 users.
  1. Gentoo/AMD64 Arch Testers
     Explains what arch testing involves and how it relates to core Gentoo/AMD64 efforts.
  2. AT Procedures
     Document explaining the formal procedures involved in arch testing for AMD64, including applications.

A. Gentoo/AMD64 Testing Docs

1. Gentoo/AMD64 Arch Testers

1.a. AMD64 Arch Testers Information

Gentoo/AMD64 will maintain a pool of "Arch Testers" hence forth referred to AT's. One or more AMD64 Developers will take the responsiblity of managing the AT group. ATs will be included on the amd64@gentoo.org alias, so they can be informed of all bug activity. The current AT head is Homer Parker. Any problems/suggestions/flames should be directed at him.

ATs will be responsible for responding to all test requests in a timely manner (keywording, security, marking stable, requests from devs on irc, etc...) and ensuring that the application works, as well as check that the ebuild doesn't contain any glaring errors. ATs must keep their system up to date, use reasonable CFLAGS, LDFLAGS, and keep with the current profile. When marking a bug as tested all ATs will use the keyword "TESTED" in bugzilla (in the "keywords" field)

Gentoo/AMD64 developers will be able to take ATs word at their own discretion. Ultimatly, commiting the requested changes will be the dev's responsibility . Feel free to ask for more than one tester if you feel the bug warrants it.

Prospective AT's will have to pass the ebuild quiz, currently here. Additionally, all quizes for Gentoo/AMD64 should be submitted to jmbsvicetoo@gentoo.org and dang@gentoo.org. For another Architectures you should submit only to jmbsvicetoo@gentoo.org.

A note about arch testers "status": Gentoo/AMD64 Arch Testers are not official Gentoo developers. They are, however, a recognized part of the Gentoo/AMD64 arch team. I ask that all AT's keep this in mind when selecting email signatures or other forms of communication.

2. AT Procedures

2.a. Marking Stable: When and How?

The imlate file is there to show which packages on amd64 are lagging behind on x86 and need to be tested and eventually marked stable. It's very useful to have people systematically testing these packages - it helps to keep amd64 as up-to-date as possible. The following guidelines are specifically aimed at Gentoo/AMD64 Arch Testers (ATs) doing testing.

• Check the newest version of the package is marked ~amd64 (TESTING), if not please read the next section of the document for the steps to follow to quickly/efficiently have a package keyworded ~amd64.
  
  
• If the package is already ~amd64, then:
  • Check to see when it was marked ~amd64. There should be a date in the package changelog, but if there isn't please use the ViewCVS functionality on http://sources.gentoo.org to look for the date it was keyworded ~amd64. If it was less than 30 days ago, the package is not eligible to be marked stable, but testing and feedback are still greatly appreciated.
  • Check for bugs on this particular version of the package. If a bug has been open in the last 30 days, it is not eligible.
  • If it has been in testing for more than 30 days, and has not had an open bug in the last 30 days, you need to test it in a thorough and systematic manner. Every conceivable permutation should be checked and rechecked - if it breaks you need to go onto Bugzilla and open a relevent bug. If it doesn't break and you are totally satisfied it's stable, continue to the final step -- but beware, its your head if this package breaks!
  • Assuming it meets all the criteria (tested for 30+ days, no bugs in the last 30 days, AT tested) you may open a bug with the title '<category>/<package>-<version> is stable on AMD64'. Assign the bug directly to amd64@gentoo.org to avoid making more work for the poor Bug Wranglers. You should include the output of your 'emerge --info' and keyword the bug STABLE. Just wait a while, a developer will review the bug, and mark stable.

2.b. Marking Testing: What's the drill?

In some cases the imlate file may show packages where the latest version is not yet ~amd64, or a new and popular package needs to be keyworded ~amd64. In these cases, we need to have them 'marked testing' - so when they're bug free for thirty days they can be made amd64 stable.

• Check the package hasn't already got any open bugs regarding it being marked testing. Quite often users will prod us about popular packages needing a keyword before any devs/ATs even notice ;)
  
  
• If the package doesn't have a bug yet, go ahead and start:
  
  
  • Check to see whether previous versions of the package were in testing or stable on AMD64. If the version increment is only a minor one (1.4.0 to 1.4.1) and previous version was stable, it's slightly different to where a package has had a major version increment or has never been ~amd64/amd64.
    
    
    • Previous version keyworded, minor version increment
      Check the changelog for the version increment, install the package and test any new, improved or otherwise modified features. Check the install is smooth, everyday functionality is there and there are no glaringly obvious bugs. If you see any bugs, file them on Bugzilla and when they're resolved test again. If everything seems okay, proceed to the final stage (putting in the ~amd64 request).
    • Previous version keyworded, major version increment
      Check the changelog, install the package and test all the new and improved features. Check for bugs in previous versions, see if they have been fixed and be especially careful to see whether new ones crept in with all the new code. Test all the other functionality, even stuff which you 'think' will work - a major version increment means a lot of changes, and it's treated almost like a new addition to the tree - everything has to be tested and verified. If you're happy it seems to be running okay, proceed to the final stage.
    • New package, not keyworded before
      Anything which has never been amd64 keyworded before is a little more tricky to process. You don't have a nice changelog to refer to for a list of things to test, a previous version which worked to use as reference or much other help. You need to install the package and then test thoroughly:
      
      
      1. Package should install without errors and be ready to run 'out of the box' with minimal effort on the part of user.
      2. Major functionality (which isn't hard to test) should all work with no significant errors. Minor errors like a typo are probably acceptable, and we understand you can't go through every operation possible, but it should work in an acceptable manner for day-to-day usage by a user.
      3. Package shouldn't break anything related...
      
      Assuming it installs, loads and works pretty well with no major errors - please proceed to the final step and congratulate your computer on adding yet another package to the expanding arsenal!
    • Package requires patches that are in bugs.gentoo.org
      Make a comment in your bug stating that these patches fix issues with the package, and CC the maintainer of the package. Developers will then wait 7-30 days to commit if maintainer does not handle the bug. The types of patches in this category include: arch specific patches, multi-lib strict, etc.
    
    
  • Assuming your testing efforts above went well, and all procedures were followed, you are now ready to open a bug and metaphysically prod a dev into committing the change.
    
    
    • Open a bug with '<category>/<package>-<version> is TESTED on AMD64' as the title. Assign the bug a keyword: TESTED
    • Assign the bug directly to amd64@gentoo.org - saves giving those Bug Wranglers yet another grey hair on their already snowy heads.
    • Include a short description of the package, what you tested and your 'emerge info'. Explicitly specify you wish the ~amd64 to be added to the keywords, it helps us grumpy old developers focus at 3am in the morning when sleep is probably a good idea ;)
    • Sit back and wait, someone will resolve the bug ASAP.

The contents of this document, unless otherwise expressly stated, are licensed under the CC-BY-SA-2.5 license. The Gentoo Name and Logo Usage Guidelines apply.