Package Building Procedures

The FreeBSD Ports Management Team

$FreeBSD: head/en_US.ISO8859-1/articles/portbuild/article.xml 40544 2013-01-05 15:22:29Z linimon $

$FreeBSD: head/en_US.ISO8859-1/articles/portbuild/article.xml 40544 2013-01-05 15:22:29Z linimon $


Table of Contents
1 Introduction
2 Build Client Management
3 Jail Build Environment Setup
4 Customizing Your Build
5 Starting the Build
6 Anatomy of a Build
7 Build Maintenance
8 Monitoring the Build
9 Dealing With Build Errors
10 Release Builds
11 Uploading Packages
12 Experimental Patches Builds
13 How to configure a new package building node
14 How to configure a new FreeBSD branch
15 How to delete an unsupported FreeBSD branch
16 How to rebase on a supported FreeBSD branch
17 How to configure a new architecture
18 How to configure a new head node (pointyhat instance)
19 Procedures for dealing with disk failures

1 Introduction

In order to provide pre-compiled binaries of third-party applications for FreeBSD, the Ports Collection is regularly built on one of the “Package Building Clusters.” Currently, the main cluster in use is at http://pointyhat.FreeBSD.org.

This article documents the internal workings of the cluster.

Note: Many of the details in this article will be of interest only to those on the Ports Management team.

1.1 The codebase

Most of the package building magic occurs under the /var/portbuild directory. Unless otherwise specified, all paths will be relative to this location. ${arch} will be used to specify one of the package architectures (e.g., amd64, arm, i386™, ia64, powerpc, SPARC64®), and ${branch} will be used to specify the build branch (e.g., 7, 7-exp, 8, 8-exp, 9, 9-exp, 10, 10-exp). The set of branches that portmgr currently supports is the same as those that the FreeBSD security team supports.

Note: Packages are no longer built for branches 4, 5, or 6, nor for the alpha architecture.

The scripts that control all of this live in /var/portbuild/scripts/. These are the checked-out copies from the Subversion repository at base/projects/portbuild/scripts/ .

Typically, incremental builds are done that use previous packages as dependencies; this takes less time, and puts less load on the mirrors. Full builds are usually only done:

  • right after release time, for the -STABLE branches

  • periodically to test changes to -CURRENT

  • for experimental ("exp-") builds

Packages from experimental builds are not uploaded.

1.2 Notes on the codebase

Until mid-2010, the scripts were completely specific to pointyhat.FreeBSD.org as the head (dispatch) node. During the summer of 2010, a significant rewrite was done in order to allow for other hosts to be head nodes. Among the changes were:

  • removal of the hard-coding of the string pointyhat

  • factoring out all configuration constants (which were previously scattered throughout the code) into configuration files (see below)

  • appending the hostname to the directories specified by buildid (this will allow directories to be unambigious when copied between machines.)

  • making the scripts more robust in terms of setting up directories and symlinks

  • where necessary, changing certain script invocations to make all the above easier

This document was originally written before these changes were made. Where things such as script invocations have changed, they were denoted as new codebase: as opposed to old codebase:.

Note: Up until November 2012, pointyhat had still been running the old codebase. That installation has now been permanently offlined. Therefore, all the instructions having to do with the old codebase have been removed.

Note: Also during this process, the codebase was migrated to the Subversion repository. For reference, the previous version may still be found in CVS.