18 How to configure a new head node (pointyhat instance)

Please talk to Mark Linimon before making any changes to this section.

18.1 Basic installation

  1. Install FreeBSD.

  2. Create a user to own the portbuild repository, such as portbuild. It should have the '*' password.

  3. Export that value for a later initialization step:

    # export PORTBUILD_USER=portbuild
    
  4. Add the following to /boot/loader.conf:

    console="vidconsole,comconsole"
    
  5. You should run the cluster on UTC. If you have not set the clock to UTC:

    # cp -p /usr/share/zoneinfo/Etc/UTC /etc/localtime
    
  6. Create the appropriate /etc/rc.conf.

    Required entries:

    hostname="${hostname}"
    sshd_enable="YES"
    

    Recommended entries:

    background_fsck="NO"
    clear_tmp_enable="YES"
    dumpdev="AUTO"
    fsck_y_enable="YES"
    
    apache22_enable="YES"
    apache_flags=""
    apache_pidfile="/var/run/httpd.pid"
    gmetad_enable="YES"
    gmond_enable="YES"
    inetd_enable="YES"
    # XXX MCL try these instead.
    inetd_flags="-R 10000 -l -w"
    mountd_enable="YES"
    nfs_server_enable="YES"
    nfs_server_flags="-u -t -n 12"
    nfs_remote_port_only="YES"
    ntpd_enable="YES"
    rpcbind_enable="YES"
    rpc_lockd_enable="NO"
    rpc_statd_enable="YES"
    sendmail_enable="NONE"
    smartd_enable="YES"
    zfs_enable="YES"
    
  7. Create /etc/resolv.conf, if necessary.

  8. Create the appropriate files in /etc/ssh/.

  9. Add the following to /etc/sysctl.conf:

    kern.maxfiles=40000
    kern.maxfilesperproc=38000
    sysctl vfs.usermount=1
    sysctl vfs.zfs.super_owner=1
    
  10. Make sure the following change is made to /etc/ttys:

    ttyu0   "/usr/libexec/getty std.9600"   vt100   on secure
    

18.2 Configuring src

You should be able to install from the most recent release using only the default kernel configuration.

18.3 Configuring ports

  1. The following ports (or their latest successors) are required:

    databases/py-sqlite3
    databases/py-sqlalchemy (only SQLITE is needed)
    devel/git (WITH_SVN)
    devel/py-configobj
    devel/py-setuptools
    devel/subversion
    net/nc
    net/rsync
    sysutils/ganglia-monitor-core (with GMETAD off)
    sysutils/ganglia-webfrontend (compile with -DWITHOUT_X11)
    www/apache22 (with EXT_FILTER)
    

    Expect those to bring in, among others:

    databases/sqlite3
    lang/perl-5.14 (or successor)
    lang/python27 (or sucessor)
    

    The following ports (or their latest successors) are strongly suggested:

    devel/ccache
    mail/postfix
    net/isc-dhcp41-server
    ports-mgmt/pkg
    ports-mgmt/portaudit
    ports-mgmt/portmaster
    security/sudo
    shells/bash
    shells/zsh
    sysutils/screen
    

    The following ports (or their latest successors) are handy:

    benchmarks/bonnie++
    ports-mgmt/pkg_tree
    sysutils/dmidecode
    sysutils/smartmontools
    sysutils/zfs-stats
    

18.4 Configuring the zfs volume and setting up the repository

The following steps need to be done as euid root.

  1. Pick a zfs volume name and export it. We have used a so far to date.

    # export ZFS_VOLUME=a
    
  2. Pick a mountpoint and export it. We have used /a so far to date.

    # export ZFS_MOUNTPOINT=/a
    
  3. Create the zfs volume and mount it.

    Example 10. Creating a zfs volume for portbuild

    # zpool create ${ZFS_VOLUME} mirror da1 da2 mirror da3 da4 mirror da5 da6 mirror da7 da8
    

    Note: We will define a zfs permission set below, so that the portbuild user may administer this volume without having to have root privileges.

  4. Select an svn repository and export it. See the FreeBSD Handbook for the currently supported list.

    # export VCS_REPOSITORY=svn://svn0.us-east.FreeBSD.org
    
  5. Obtain a copy of the kickstart script into a temporary directory. (You will not need to keep this directory later.)

    # mkdir -p /home/portbuild/tmp
    # svn checkout ${VCS_REPOSITORY}/base/projects/portbuild/tools /home/portbuild/tmp
    
  6. Run the kickstart script:

    # sh /home/portbuild/tmp/mkportbuild
    

    This will accomplish all the following 5 steps:

    1. Create the portbuild directory:

      # mkdir -p ${ZFS_MOUNTPOINT}/portbuild
      
    2. Create and mount a new zfs filesystem on it:

      zfs create -o mountpoint=${ZFS_MOUNTPOINT}/portbuild ${ZFS_VOLUME}/portbuild
      
    3. Set up the directory:

      # chown ${PORTBUILD_USER}:${PORTBUILD_USER} ${ZFS_MOUNTPOINT}/portbuild
      # chmod 775 ${ZFS_MOUNTPOINT}/portbuild
      # ln -sf ${ZFS_MOUNTPOINT}/portbuild /var/portbuild
      

      Note: The ln is necessary due to a number of hardcoded paths. This is a bug.

    4. set up the repository:

      % svn checkout ${VCS_REPOSITORY}/base/projects/portbuild ${ZFS_MOUNTPOINT}/portbuild
      
    5. Set up the zfs permission set:

      # ${ZFS_MOUNTPOINT}/tools/zfsadmin
      

18.5 Configuring the portbuild files

  1. Configure how build slaves will talk to your server by making the following changes to /a/portbuild/conf/client.conf:

    • Set CLIENT_NFS_MASTER to wherever your build slaves will PXE boot from. (Possibly, the hostname of your server.)

    • Set CLIENT_BACKUP_FTP_SITE to a backup site for FTP fetches; again, possibly the hostname of your server.

    • Set CLIENT_UPLOAD_HOST to where completed packages will be uploaded.

    Most of the other default values should be fine.

  2. Most of the default values in /a/portbuild/conf/common.conf should be fine. This file holds definitions used by both the server and all its clients.

  3. Configure the server by making the following changes to /a/portbuild/conf/server.conf:

    • Set SUPPORTED_ARCHS to the list of architectures you wish to build packages for.

    • For each source branch you will be building for, set SRC_BRANCHES and SRC_BRANCH_branch_SUBDIR as detailed in Section 14.1. You should not need to change SRC_BRANCHES_PATTERN.

    • Set ZFS_VOLUME and ZFS_MOUNTPOINT to whatever you chose above.

    • Set UPLOAD_DIRECTORY, UPLOAD_TARGET, and UPLOAD_USER as appropriate for your site.

    • Set VCS_REPOSITORY to whatever you chose above.

    • Set MASTER_URL to the http URL of your server. This will be stamped into the package build logs and the indices thereof.

    Most of the other default values should be fine.

18.6 pre-qmanager

  1. For each architecture, follow the steps in Section 17.1.

18.7 qmanager

  1. Copy the following files from /var/portbuild/etc/rc.d/ to /usr/local/etc/rc.d/:

    buildproxy
    pollmachine
    qmanager
    

    As root, start each one of them. You may find it handy to start each under screen for debugging purposes.

  2. Initialize the qmanager database's acl list:

    Note: This should now be automatically done for you by the first build command.

    # python /a/portbuild/qmanager/qclient add_acl name=deny_all uidlist= gidlist= sense=0
    

18.8 Creating src and ports repositories

  1. As the portbuild user, run the following commands manually to create the src and ports repositories, respectively:

    % /a/portbuild/scripts/updatesnap.ports
    % /a/portbuild/scripts/updatesnap
    

    These will be periodically run from the portbuild crontab, which you will install below.

18.9 Other services

  1. Configure /usr/local/etc/apache22/httpd.conf as appropriate for your site.

  2. Copy /var/portbuild/conf/apache.conf to the appropriate Includes/ subdirectory, e.g., /usr/local/etc/apache22/Includes/portbuild.conf. Configure it as appropriate for your site.

  3. Install /var/portbuild/crontabs/root as the root crontab via crontab -e.

  4. Install /var/portbuild/crontabs/portbuild as the portbuild crontab via crontab -u portbuild -e. If you do not support all the archs listed there, make sure to comment out the appropriate dologs entries.

  5. If your build slaves will be pxebooted, make sure to enable the tftp entries in /etc/inetd.conf.

  6. Configure mail by doing the following:

    newaliases.

18.10 Finishing up

  1. For each architecture, follow the steps in Section 17.2.

  2. At some point, you will probably find it handy to append the following to the PATH definition for the portbuild user:

    /a/portbuild/scripts:/a/portbuild/tools
    

You should now be ready to build packages.