Project:Infrastructure/Mirrors/Distfile Mirroring System
This guide describes how our distfile process flow works, including how a new tarball gets added to the distfile mirrors
Placing files on the Gentoo Mirror system
Historically Gentoo developers have utilized dev.gentoo.org:/space/distfiles-local which then allowed the usage of mirror://gentoo/ in ebuilds, but now the proper way to manually mirroring distfiles is to place it in their ~/public_html/ space on dev.gentoo.org and putting that to ebuild's SRC_URI entry. See devmanual entry as a concrete example.
Try to use common naming conventions to avoid conflicts with distfiles, e.g. ${P}.tar.xz since the mirror system only downloads the first instance of a file name. If subsequent ebuilds reference this file name the checksums of the two URI's are compared, if they do not match the second file will not be fetched. The mirror system will produce an error and human intervention is required. Please check file names carefully.
Common fetch errors:
- URI port must be 80, 443, or 23
- URI is malformed (mirrors:// is a common mistake, mirror:// is proper)
- Mirror target isn't valid (doesn't specify a valid tier)
- Checksum conflict with another ebuild in the tree - check your file name
- Upstream host timeout while attempting to connect - Mirror System will reattempt at next pass
- Upstream host isn't valid - check your URL name.
Technical Details
master private distfile mirror
The below is most likely outdated.
Every hour, a script that roughly contains this command is ran.
/usr/bin/emirrordist \
--distfiles=${DATADIR}/distfiles/ \
--delete --jobs=10 --repo=gentoo \
--deletion-delay=${DELAY} \
--failure-log=${LOGDIR}/failure.log \
--success-log=${LOGDIR}/success.log \
--scheduled-deletion-log=${LOGDIR}/deletion.log \
--deletion-db=${LOGDIR}/deletion-db.bdb \
--distfiles-db=${LOGDIR}/distfile-db.bdb \
--temp-dir=${DATADIR}/tmp/ \
--whitelist-from=${DATADIR}/tmp/whitelist-master.txt \
--distfiles-local=${DATADIR}/distfiles-local \
--mirror