The second part of the Makefile describes the files that must be downloaded in order to build the port, and where they can be downloaded from.
DISTNAME is the name of the port as called by the authors of the software. DISTNAME defaults to ${PORTNAME}-${PORTVERSION}, so override it only if necessary. DISTNAME is only used in two places. First, the distribution file list (DISTFILES) defaults to ${DISTNAME}${EXTRACT_SUFX}. Second, the distribution file is expected to extract into a subdirectory named WRKSRC, which defaults to work/${DISTNAME}.
Some vendor's distribution names which do not fit into the ${PORTNAME}-${PORTVERSION}-scheme can be handled automatically by setting DISTVERSION. PORTVERSION and DISTNAME will be derived automatically, but can of course be overridden. The following table lists some examples:
Note: PKGNAMEPREFIX and PKGNAMESUFFIX do not affect DISTNAME. Also note that if WRKSRC is equal to work/${PORTNAME}-${PORTVERSION} while the original source archive is named something other than ${PORTNAME}-${PORTVERSION}${EXTRACT_SUFX}, you should probably leave DISTNAME alone— you are better off defining DISTFILES than having to set both DISTNAME and WRKSRC (and possibly EXTRACT_SUFX).
Record the directory part of the FTP/HTTP-URL pointing at the original tarball in MASTER_SITES. Do not forget the trailing slash (/)!
The make macros will try to use this specification for grabbing the distribution file with FETCH if they cannot find it already on the system.
It is recommended that you put multiple sites on this list, preferably from different continents. This will safeguard against wide-area network problems. We are even planning to add support for automatically determining the closest master site and fetching from there; having multiple sites will go a long way towards helping this effort.
If the original tarball is part of one of the popular archives such as SourceForge, GNU, or Perl CPAN, you may be able refer to those sites in an easy compact form using MASTER_SITE_* (e.g., MASTER_SITE_SOURCEFORGE, MASTER_SITE_GNU and MASTER_SITE_PERL_CPAN). Simply set MASTER_SITES to one of these variables and MASTER_SITE_SUBDIR to the path within the archive. Here is an example:
MASTER_SITES= ${MASTER_SITE_GNU}
MASTER_SITE_SUBDIR= make
Or you can use a condensed format:
MASTER_SITES= GNU/make
These variables are defined in /usr/ports/Mk/bsd.sites.mk. There are new entries added all the time, so make sure to check the latest version of this file before submitting a port.
Several magic macros exist for popular sites with a predictable directory structure. For these, just use the abbreviation and the system will try to guess the correct subdirectory for you.
MASTER_SITES= SF
If the guess is incorrect, it can be overridden as follows.
MASTER_SITES= SF/stardict/WyabdcRealPeopleTTS/${PORTVERSION}
This can be also written as
MASTER_SITES= SF
MASTER_SITE_SUBDIR= stardict/WyabdcRealPeopleTTS/${PORTVERSION}
Table 5-1. Popular Magic MASTER_SITES Macros
| Macro | Assumed subdirectory |
|---|---|
| APACHE_JAKARTA | /dist/jakarta/${PORTNAME:S,-,,/,}/source |
| BERLIOS | /${PORTNAME:L} |
| CHEESESHOP | /packages/source/source/${DISTNAME:C/(.).*/\1/}/${DISTNAME:C/(.*)-[0-9].*/\1/} |
| DEBIAN | /debian/pool/main/${PORTNAME:C/^((lib)?.).*$/\1/}/${PORTNAME} |
| GCC | /pub/gcc/releases/${DISTNAME} |
| GNOME | /pub/GNOME/sources/${PORTNAME}/${PORTVERSION:C/^([0-9]+\.[0-9]+).*/\1/} |
| GNU | /gnu/${PORTNAME} |
| MOZDEV | /pub/mozdev/${PORTNAME:L} |
| PERL_CPAN | /pub/CPAN/modules/by-module/${PORTNAME:C/-.*//} |
| PYTHON | /ftp/python/${PYTHON_PORTVERSION:C/rc[0-9]//} |
| RUBYFORGE | /${PORTNAME:L} |
| SAVANNAH | /${PORTNAME:L} |
| SF | /project/${PORTNAME:L}/${PORTNAME:L}/${PORTVERSION} |
If you have one distribution file, and it uses an odd suffix to indicate the compression mechanism, set EXTRACT_SUFX.
For example, if the distribution file was named foo.tgz instead of the more normal foo.tar.gz, you would write:
DISTNAME= foo EXTRACT_SUFX= .tgz
The USE_BZIP2, USE_XZ and USE_ZIP variables automatically set EXTRACT_SUFX to .tar.bz2, .tar.xz or .zip as necessary. If neither of these are set then EXTRACT_SUFX defaults to .tar.gz.
Note: You never need to set both EXTRACT_SUFX and DISTFILES.
Sometimes the names of the files to be downloaded have no resemblance to the name of the port. For example, it might be called source.tar.gz or similar. In other cases the application's source code might be in several different archives, all of which must be downloaded.
If this is the case, set DISTFILES to be a space separated list of all the files that must be downloaded.
DISTFILES= source1.tar.gz source2.tar.gz
If not explicitly set, DISTFILES defaults to ${DISTNAME}${EXTRACT_SUFX}.
If only some of the DISTFILES must be extracted—for example, one of them is the source code, while another is an uncompressed document—list the filenames that must be extracted in EXTRACT_ONLY.
DISTFILES= source.tar.gz manual.html EXTRACT_ONLY= source.tar.gz
If none of the DISTFILES should be uncompressed then set EXTRACT_ONLY to the empty string.
EXTRACT_ONLY=
If your port requires some additional patches that are available by FTP or HTTP, set PATCHFILES to the names of the files and PATCH_SITES to the URL of the directory that contains them (the format is the same as MASTER_SITES).
If the patch is not relative to the top of the source tree (i.e., WRKSRC) because it contains some extra pathnames, set PATCH_DIST_STRIP accordingly. For instance, if all the pathnames in the patch have an extra foozolix-1.0/ in front of the filenames, then set PATCH_DIST_STRIP=-p1.
Do not worry if the patches are compressed; they will be decompressed automatically if the filenames end with .gz or .Z.
If the patch is distributed with some other files, such as documentation, in a gzipped tarball, you cannot just use PATCHFILES. If that is the case, add the name and the location of the patch tarball to DISTFILES and MASTER_SITES. Then, use the EXTRA_PATCHES variable to point to those files and bsd.port.mk will automatically apply them for you. In particular, do not copy patch files into the PATCHDIR directory—that directory may not be writable.
Note: The tarball will have been extracted alongside the regular source by then, so there is no need to explicitly extract it if it is a regular gzipped or compressed tarball. If you do the latter, take extra care not to overwrite something that already exists in that directory. Also, do not forget to add a command to remove the copied patch in the pre-clean target.
(Consider this to be a somewhat ``advanced topic''; those new to this document may wish to skip this section at first).
This section has information on the fetching mechanism known as both MASTER_SITES:n and MASTER_SITES_NN. We will refer to this mechanism as MASTER_SITES:n.
A little background first. OpenBSD has a neat feature inside the DISTFILES and PATCHFILES variables which allows files and patches to be postfixed with :n identifiers. Here, n can be both [0-9] and denote a group designation. For example:
DISTFILES= alpha:0 beta:1
In OpenBSD, distribution file alpha will be associated with variable MASTER_SITES0 instead of our common MASTER_SITES and beta with MASTER_SITES1.
This is a very interesting feature which can decrease that endless search for the correct download site.
Just picture 2 files in DISTFILES and 20 sites in MASTER_SITES, the sites slow as hell where beta is carried by all sites in MASTER_SITES, and alpha can only be found in the 20th site. It would be such a waste to check all of them if the maintainer knew this beforehand, would it not? Not a good start for that lovely weekend!
Now that you have the idea, just imagine more DISTFILES and more MASTER_SITES. Surely our ``distfiles survey meister'' would appreciate the relief to network strain that this would bring.
In the next sections, information will follow on the FreeBSD implementation of this idea. We improved a bit on OpenBSD's concept.
This section tells you how to quickly prepare fine grained fetching of multiple distribution files and patches from different sites and subdirectories. We describe here a case of simplified MASTER_SITES:n usage. This will be sufficient for most scenarios. However, if you need further information, you will have to refer to the next section.
Some applications consist of multiple distribution files that must be downloaded from a number of different sites. For example, Ghostscript consists of the core of the program, and then a large number of driver files that are used depending on the user's printer. Some of these driver files are supplied with the core, but many others must be downloaded from a variety of different sites.
To support this, each entry in DISTFILES may be followed by a colon and a ``tag name''. Each site listed in MASTER_SITES is then followed by a colon, and the tag that indicates which distribution files should be downloaded from this site.
For example, consider an application with the source split in two parts, source1.tar.gz and source2.tar.gz, which must be downloaded from two different sites. The port's Makefile would include lines like Example 5-1.
Example 5-1. Simplified Use of MASTER_SITES:n with One File Per Site
MASTER_SITES= ftp://ftp.example1.com/:source1 \ ftp://ftp.example2.com/:source2 DISTFILES= source1.tar.gz:source1 \ source2.tar.gz:source2
Multiple distribution files can have the same tag. Continuing the previous example, suppose that there was a third distfile, source3.tar.gz, that should be downloaded from ftp.example2.com. The Makefile would then be written like Example 5-2.
Okay, so the previous section example did not reflect your needs? In this section we will explain in detail how the fine grained fetching mechanism MASTER_SITES:n works and how you can modify your ports to use it.
Elements can be postfixed with :n where n is [^:,]+, i.e., n could conceptually be any alphanumeric string but we will limit it to [a-zA-Z_][0-9a-zA-Z_]+ for now.
Moreover, string matching is case sensitive; i.e., n is different from N.
However, the following words cannot be used for postfixing purposes since they yield special meaning: default, all and ALL (they are used internally in item ii). Furthermore, DEFAULT is a special purpose word (check item 3).
Elements postfixed with :n belong to the group n, :m belong to group m and so forth.
Elements without a postfix are groupless, i.e., they all belong to the special group DEFAULT. If you postfix any elements with DEFAULT, you are just being redundant unless you want to have an element belonging to both DEFAULT and other groups at the same time (check item 5).
The following examples are equivalent but the first one is preferred:
MASTER_SITES= alpha
MASTER_SITES= alpha:DEFAULT
Groups are not exclusive, an element may belong to several different groups at the same time and a group can either have either several different elements or none at all. Repeated elements within the same group will be simply that, repeated elements.
When you want an element to belong to several groups at the same time, you can use the comma operator (,).
Instead of repeating it several times, each time with a different postfix, we can list several groups at once in a single postfix. For instance, :m,n,o marks an element that belongs to group m, n and o.
All the following examples are equivalent but the last one is preferred:
MASTER_SITES= alpha alpha:SOME_SITE
MASTER_SITES= alpha:DEFAULT alpha:SOME_SITE
MASTER_SITES= alpha:SOME_SITE,DEFAULT
MASTER_SITES= alpha:DEFAULT,SOME_SITE
All sites within a given group are sorted according to MASTER_SORT_AWK. All groups within MASTER_SITES and PATCH_SITES are sorted as well.
Group semantics can be used in any of the following variables MASTER_SITES, PATCH_SITES, MASTER_SITE_SUBDIR, PATCH_SITE_SUBDIR, DISTFILES, and PATCHFILES according to the following syntax:
All MASTER_SITES, PATCH_SITES, MASTER_SITE_SUBDIR and PATCH_SITE_SUBDIR elements must be terminated with the forward slash / character. If any elements belong to any groups, the group postfix :n must come right after the terminator /. The MASTER_SITES:n mechanism relies on the existence of the terminator / to avoid confusing elements where a :n is a valid part of the element with occurrences where :n denotes group n. For compatibility purposes, since the / terminator was not required before in both MASTER_SITE_SUBDIR and PATCH_SITE_SUBDIR elements, if the postfix immediate preceding character is not a / then :n will be considered a valid part of the element instead of a group postfix even if an element is postfixed with :n. See both Example 5-3 and Example 5-4.
Example 5-3. Detailed Use of MASTER_SITES:n in MASTER_SITE_SUBDIR
MASTER_SITE_SUBDIR= old:n new/:NEW
Directories within group DEFAULT -> old:n
Directories within group NEW -> new
Example 5-4. Detailed Use of MASTER_SITES:n with Comma Operator, Multiple Files, Multiple Sites and Multiple Subdirectories
MASTER_SITES= http://site1/%SUBDIR%/ http://site2/:DEFAULT \ http://site3/:group3 http://site4/:group4 \ http://site5/:group5 http://site6/:group6 \ http://site7/:DEFAULT,group6 \ http://site8/%SUBDIR%/:group6,group7 \ http://site9/:group8 DISTFILES= file1 file2:DEFAULT file3:group3 \ file4:group4,group5,group6 file5:grouping \ file6:group7 MASTER_SITE_SUBDIR= directory-trial:1 directory-n/:groupn \ directory-one/:group6,DEFAULT \ directory
The previous example results in the following fine grained fetching. Sites are listed in the exact order they will be used.
file1 will be fetched from
MASTER_SITE_OVERRIDE
http://site1/directory-trial:1/
http://site1/directory-one/
http://site1/directory/
http://site2/
http://site7/
MASTER_SITE_BACKUP
file2 will be fetched exactly as file1 since they both belong to the same group
MASTER_SITE_OVERRIDE
http://site1/directory-trial:1/
http://site1/directory-one/
http://site1/directory/
http://site2/
http://site7/
MASTER_SITE_BACKUP
file3 will be fetched from
MASTER_SITE_OVERRIDE
http://site3/
MASTER_SITE_BACKUP
file4 will be fetched from
MASTER_SITE_OVERRIDE
http://site4/
http://site5/
http://site6/
http://site7/
http://site8/directory-one/
MASTER_SITE_BACKUP
file5 will be fetched from
MASTER_SITE_OVERRIDE
MASTER_SITE_BACKUP
file6 will be fetched from
MASTER_SITE_OVERRIDE
http://site8/
MASTER_SITE_BACKUP
How do I group one of the special variables from bsd.sites.mk, e.g., MASTER_SITE_SOURCEFORGE?
See Example 5-5.
Example 5-5. Detailed Use of MASTER_SITES:n with MASTER_SITE_SOURCEFORGE
MASTER_SITES= http://site1/ ${MASTER_SITE_SOURCEFORGE:S/$/:sourceforge,TEST/}
DISTFILES= something.tar.gz:sourceforge
something.tar.gz will be fetched from all sites within MASTER_SITE_SOURCEFORGE.
How do I use this with PATCH* variables?
All examples were done with MASTER* variables but they work exactly the same for PATCH* ones as can be seen in Example 5-6.
All current ports remain the same. The MASTER_SITES:n feature code is only activated if there are elements postfixed with :n like elements according to the aforementioned syntax rules, especially as shown in item 7.
The port targets remain the same: checksum, makesum, patch, configure, build, etc. With the obvious exceptions of do-fetch, fetch-list, master-sites and patch-sites.
do-fetch: deploys the new grouping postfixed DISTFILES and PATCHFILES with their matching group elements within both MASTER_SITES and PATCH_SITES which use matching group elements within both MASTER_SITE_SUBDIR and PATCH_SITE_SUBDIR. Check Example 5-4.
fetch-list: works like old fetch-list with the exception that it groups just like do-fetch.
master-sites and patch-sites: (incompatible with older versions) only return the elements of group DEFAULT; in fact, they execute targets master-sites-default and patch-sites-default respectively.
Furthermore, using target either master-sites-all or patch-sites-all is preferred to directly checking either MASTER_SITES or PATCH_SITES. Also, directly checking is not guaranteed to work in any future versions. Check item iii.ii for more information on these new port targets.
New port targets
There are master-sites-n and patch-sites-n targets which will list the elements of the respective group n within MASTER_SITES and PATCH_SITES respectively. For instance, both master-sites-DEFAULT and patch-sites-DEFAULT will return the elements of group DEFAULT, master-sites-test and patch-sites-test of group test, and thereon.
There are new targets master-sites-all and patch-sites-all which do the work of the old master-sites and patch-sites ones. They return the elements of all groups as if they all belonged to the same group with the caveat that it lists as many MASTER_SITE_BACKUP and MASTER_SITE_OVERRIDE as there are groups defined within either DISTFILES or PATCHFILES; respectively for master-sites-all and patch-sites-all.
Do not let your port clutter /usr/ports/distfiles. If your port requires a lot of files to be fetched, or contains a file that has a name that might conflict with other ports (e.g., Makefile), set DIST_SUBDIR to the name of the port (${PORTNAME} or ${PKGNAMEPREFIX}${PORTNAME} should work fine). This will change DISTDIR from the default /usr/ports/distfiles to /usr/ports/distfiles/DIST_SUBDIR, and in effect puts everything that is required for your port into that subdirectory.
It will also look at the subdirectory with the same name on the backup master site at ftp.FreeBSD.org. (Setting DISTDIR explicitly in your Makefile will not accomplish this, so please use DIST_SUBDIR.)
Note: This does not affect the MASTER_SITES you define in your Makefile.
If your port uses binary distfiles and has a license that requires that the source code is provided with packages distributed in binary form, e.g., GPL, ALWAYS_KEEP_DISTFILES will instruct the FreeBSD build cluster to keep a copy of the files specified in DISTFILES. Users of these ports will generally not need these files, so it is a good idea to only add the source distfiles to DISTFILES when PACKAGE_BUILDING is defined.
Example 5-7. Use of ALWAYS_KEEP_DISTFILES
.if defined(PACKAGE_BUILDING) DISTFILES+= foo.tar.gz ALWAYS_KEEP_DISTFILES= yes .endif
When adding extra files to DISTFILES, make sure you also add them to distinfo. Also, the additional files will normally be extracted into WRKDIR as well, which for some ports may lead to undesirable side effects and require special handling.