FreeBSD 5.4-RELEASE Migration Guide

The FreeBSD Release Engineering Team

Copyright © 2002, 2003, 2004, 2005 The FreeBSD Release Engineering Team

$FreeBSD: src/release/doc/en_US.ISO8859-1/early-adopter/article.sgml,v 1.21.2.7 2005/03/25 15:53:33 bmah Exp $

FreeBSD is a registered trademark of the FreeBSD Foundation.

Apple, FireWire, Mac, Macintosh, Mac OS, Quicktime, and TrueType are trademarks of Apple Computer, Inc., registered in the United States and other countries.

IBM, AIX, EtherJet, Netfinity, OS/2, PowerPC, PS/2, S/390, and ThinkPad are trademarks of International Business Machines Corporation in the United States, other countries, or both.

IEEE, POSIX, and 802 are registered trademarks of Institute of Electrical and Electronics Engineers, Inc. in the United States.

Intel, Celeron, EtherExpress, i386, i486, Itanium, Pentium, and Xeon are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

Microsoft, IntelliMouse, MS-DOS, Outlook, Windows, Windows Media and Windows NT are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries.

Sparc, Sparc64, SPARCEngine, and UltraSPARC are trademarks of SPARC International, Inc in the United States and other countries. Products bearing SPARC trademarks are based upon architecture developed by Sun Microsystems, Inc.

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this document, and the FreeBSD Project was aware of the trademark claim, the designations have been followed by the ``™'' or the ``®'' symbol.

This article describes major differences between FreeBSD 4.X and FreeBSD 5.4-RELEASE, from the viewpoint of a user or administrator upgrading an existing system. It provides a brief overview of FreeBSD release engineering. It then describes some of the new features in FreeBSD 5.X, as well as some changes that might be of particular interest to users accustomed to working with the 4.X series of releases. This article also describes binary- and source-based upgrading procedures for existing systems. Earlier versions of this article appeared in prior FreeBSD 5.X releases. Prior to FreeBSD 5.3-RELEASE, this article was known as the ``Early Adopters Guide''.

1 Introduction

FreeBSD 5.4-RELEASE is the latest version of FreeBSD in the ``FreeBSD-STABLE'' series of releases. This and future releases in the 5.X series are targeted toward production usage in much the same way as the prior 4.X series of releases.

This article addresses a number of topics and issues of interest to users updating from a 4.X release to 5.4-RELEASE. It begins with a brief overview of current release engineering practices and then describes some of the new features available with the FreeBSD 5.X series. Perhaps the most important section lists issues where major changes have taken place, user-visible behavior has changed, or external software interfaces have been modified. Last are some notes on upgrading existing FreeBSD 4.X systems to FreeBSD 5.4-RELEASE, from binaries or from source.

2 An Overview of the FreeBSD Release Process

FreeBSD employs a development model relying on multiple branches within the source code repository. The main branch is called ``CURRENT'', and is referred to in the CVS repository with the HEAD tag. New features are committed first to this branch; although CURRENT is the first to see new functionality, it also occasionally suffers from breakages as new features are added and debugged.

Most FreeBSD releases are made from one of several ``STABLE'' branches. Features are added to these branches only after some amount of testing in CURRENT. For the past two years, the only STABLE branch under active development was known as ``4-STABLE'', and all of the FreeBSD 4.X releases were based on it. This branch has the tag RELENG_4 in the CVS repository.

The past two stable branches (3-STABLE and 4-STABLE) were created very early in their development cycles (their branchpoints were 3.1 and 4.0, respectively). In hindsight, this practice did not give sufficient time for either CURRENT to stabilize before the new branches were created. This in turn resulted in wasted effort porting bug fixes between branches, as well as creating some architectural changes that could not be ported between branches at all.

The 5.X releases have been handled slightly differently. FreeBSD 5.0, 5.1, and 5.2 were based on the CURRENT branch. The first of these releases was made after over two years of development (prior to these, the last release from HEAD was FreeBSD 4.0, in March 2000).

Unlike prior ``STABLE'' branches, the ``5-STABLE'' branch in CVS (with the branch tag RELENG_5) was created more than a year after the first 5.X release (specifically, during the 5.3 release cycle). This delay gave time for the development team to complete needed architectural changes, stabilize the system, finalize various interfaces, and create a good starting point for the remaining 5.X releases.

Aside from general improvements and bug fixes, a major priority for development on the 5-STABLE branch is the preservation of Application Binary Interface (ABI) and Application Program Interface (API) compatibility. Any changes that could break backward compatibility (including kernel or library interfaces) are strongly discouraged, and will not be permitted except as a last-resort solution to a critical problem.

The next release from CURRENT will likely be 6.0-RELEASE, created from CVS HEAD. There is no firm date for 6.0, as of this writing, although it is expected sometime in the third quarter of 2005. Future release engineering plans call for new major releases approximately every eighteen months, with minor releases approximately every four months.

The last planned release along the 4-STABLE development branch was 4.11-RELEASE, which was announced in January 2005. No new major features are planned for the 4-STABLE branch at this time, and any future work will likely be limited to bug fixes and security patches. For especially conservative users, it may be reasonable to continue using the 4.X releases for a time. The Security Officer Team will continue to support releases made from the 4-STABLE branch in accordance with their published policies, which can be found on the Security page on the FreeBSD web site.

More information on FreeBSD release engineering processes can be found on the Release Engineering Web pages and in the ``FreeBSD Release Engineering'' article.

3 New Features

A major attraction of FreeBSD 5.X is a number of new features, generally involving large architectural changes that were not feasible to port back to the FreeBSD 4-STABLE development branch. (By contrast, many self-contained enhancements, such as new device drivers or updates to userland utilities, have already been ported.) A brief, but not exhaustive list includes:

A more comprehensive list of new features can be found in the release notes for the various FreeBSD 5.X releases.

4 Notable Changes

Some of the differences between FreeBSD 4.X and 5.X deserve special mention because they involve major architectural changes, or break backwards compatibility in some way. While these changes are unlikely to cause a loss of data, they could cause some confusion for the unwary. Some notable examples are:

More information can be found in the release notes for the various FreeBSD 5.X releases, as well as the src/UPDATING file in the FreeBSD source tree. Notable changes to the FreeBSD Ports Collection are listed in the ports/UPDATING and ports/CHANGES files in the ports tree.

5 Notes on Upgrading from FreeBSD 4.X

For those users with existing FreeBSD systems, this section offers a few notes on upgrading a FreeBSD 4.X system to 5.X. As with any FreeBSD upgrade, it is crucial to read the release notes and the errata for the version in question, as well as src/UPDATING in the case of source upgrades.

5.1 Binary Upgrades

Probably the most straightforward approach is that of ``backup everything, reformat, reinstall, and restore user data''. This eliminates problems of incompatible or obsolete executables and configuration files polluting the new system. It allows new file systems to be created to take advantage of new functionality (most notably, the UFS2 defaults).

As of this writing, the binary upgrade option in sysinstall(8) has not been well-tested for cross-major-version upgrades. Use of this feature is not recommended.

Several changes may be of interest to those users accustomed to the FreeBSD 4.X boot media. Installation floppies (on platforms that support them, such as i386), are organized somewhat differently than on prior releases. On 4.X releases, the floppy set contained a stripped-down kernel with just enough functionality to install the system. This arrangement allowed the kernel to fit on a single floppy disk, but it lacked the device drivers required by certain hardware configurations. Beginning with FreeBSD 5.3-RELEASE, the installation floppies contain a standard GENERIC kernel segmented across multiple disks, with a much more complete set of drivers and features. The boot loader prompts for the insertion of additional disks as required. Users downloading floppy images (perhaps to perform a network-based installation) should note that the floppy disk set now includes three disks: boot.flp, kern1.flp, and kern2.flp.

The installation CDROMs for the i386 architecture now use a ``no-emulation'' boot loader. This allows, among other things, the use of a GENERIC kernel, rather than the stripped-down kernels used on the floppy images in previous versions. Generally, any system capable of booting the Microsoft Windows NT® 4, Windows 2000, or Windows XP installation CDROMs should be able to boot the FreeBSD 5.X CDROMs.

The layout of CDROMs has changed from the 4.X series. Beginning with 5.4-RELEASE, the installation image and the ``live filesystem'' are now contained on the first disk on all architectures except for ia64 (ia64 releases use separate installation and live filesystem disks due to disk space constraints). Common packages are now contained on the second and succeeding disks of a release.

5.2 Source Upgrades

Note: Many users and developers have found it easier to backup all their data and configuration files (a wise precaution in any case), perform a binary installation (such as from CDROM), and restore their data. Compared to a source upgrade, the binary upgrade removes the need to deal with leftover files and programs on the disk, and allows the system to take advantage of new file system features such as the UFS2 file system layout.

Users unfamiliar with the buildworld/installworld procedures for updating FreeBSD from source should not attempt a source upgrade, but should instead perform a binary installation after backing up all data.

A source-based upgrade procedure builds and installs a set of binaries compiled from source on the local machine. It is based on the buildworld/installworld procedure often used by advanced FreeBSD users to track changes along a development branch (such as FreeBSD-STABLE or FreeBSD-CURRENT). In general, this procedure involves more effort than the binary upgrade procedure, but may be useful when a system's configuration files are complex or have been highly customized. A source upgrade can also be useful for a remote system where an administrator has remote console access but no physical access (and therefore cannot insert installation media).

Reading src/UPDATING is absolutely essential. The section entitled ``To upgrade in-place from 4.x-stable to current'' contains a step-by-step update procedure, which must be followed exactly, without making use of the ``shortcuts'' that some users occasionally employ. An annotated list of these steps is presented below:

  1. Make backups.

    The importance of this step cannot be overstated. It is important to make backups of all user data and configuration files. Level zero dumps with dump(8) are an excellent way to do this, although there are of course many workable alternatives.

  2. Ensure that there is about 30MB of free disk space on the / file system.

    FreeBSD 5.X uses more space than FreeBSD 4.X. If the /tmp directory resides on the / file system (as is frequently the case), deleting all of the content from this directory may help to free up needed space.

  3. Fix /etc/fstab if required.

    This item probably only affects older FreeBSD/i386 systems. On systems that use MBR-style disk slices, FreeBSD used to support ``compatibility slices'', where disk partition names could take the form /dev/ad0a (without specifying a slice name explicitly). These are no longer supported; disk partitions must be named according to the form /dev/ad0s1a. In other words, disk partitions must now completely specify a disk device, a slice number, and a partition letter.

    Note that ``compatibility slices'' have generally not been used by default since FreeBSD 2.2.6-RELEASE. This item does not apply to FreeBSD/alpha at all, or to systems using ``dangerously dediated'' mode.

  4. Using cvs(1) or other means, obtain a copy of the FreeBSD base system sources (the src/ module). To check out 5.4-RELEASE from the FreeBSD CVS repository, use the RELENG_5_4_0_RELEASE tag. For the 5.4 release and errata branch, use the RELENG_5_4 tag. To track the FreeBSD 5-STABLE development branch, use the RELENG_5 branch tag. When using CVS to check out the source tree, it is important to pass the -P flag to CVS so that it prunes away empty directories.

  5. FreeBSD 5.X requires some new entries in the system password and group files, corresponding to various system services. They should be installed by running:

    # mergemaster -p

    This step must be performed to give some new files the correct usernames and groupnames.

    Note that in FreeBSD 5.X, the location of the nologin(8) utility has moved from /sbin/nologin to /usr/sbin/nologin. Because a number of pseudo-users have nologin(8) as their login shell, this change generates additional differences in /etc/passwd.

  6. Build the new userland using:

    # cd /usr/src
# make buildworld

    If CPUTYPE is defined in /etc/make.conf, it should be defined using the ?= operator, so the buildworld process can override this variable if necessary.

    Note that the MAKEOBJDIRPREFIX must not be defined in /etc/make.conf.

    In general, most of the build ``knobs'' defined in /etc/make.conf should be commented out or removed. This statement especially applies to those that directly affect the build or basic operation of FreeBSD, such as NO_TOOLCHAIN.

  7. Build and install a new kernel using:

    # make buildkernel

    Note that the buildkernel make(1) target must be used in order to ensure that the resulting kernel is compiled with the toolchain built in the buildworld step above. Manually using config(8) to set up a kernel build area and attempting to build a kernel will not work.

    Although building (and later installing) a custom kernel at this point is feasible, upgrading using the GENERIC kernel and installing a custom kernel configuration later may be less error-prone. When trying to build a custom kernel for the first time, using the GENERIC kernel from FreeBSD 5.X as a template is highly recommended, due to the number of device and options that have been added or changed since 4.X. In any case, including the COMPAT_FREEBSD4 kernel configuration option is crucial for a successful upgrade.

  8. Set up a device hints file for ISA devices with a variant of the following command. Substitute the appropriate architecture name (e.g. i386) for MACHINE.

    # cp sys/MACHINE/conf/GENERIC.hints /boot/device.hints

    While an empty device hint file may be sufficient for modern hardware, ISA hints are needed for systems with custom ISA non-PNP cards (with custom hints being needed), for non-PNPBIOS systems, or for floppy disks to work correctly on PNPBIOS systems.

  9. Install the new kernel with:

    # make installkernel

    Note that while the FreeBSD 4.X kernel installs to /kernel by default, the FreeBSD 5.X kernel installs to /boot/kernel/kernel. Kernel modules under 4.X normally install to the /modules directory, whereas under 5.X they install to /boot/kernel.

  10. Install the FreeBSD 5.X boot loader with:

    # cd /usr/src/sys/boot
# make STRIP="" install

    This step, though optional, is highly recommended.

  11. Disable third-party modules (such as those for VMware) to prevent crashes caused by changes in kernel ABIs or other incompatibilities.

  12. Reboot to single-user mode. Rebooting at this point is absolutely required because the new kernel must be running to install the new userland. Being in single-user mode will drastically decrease the potential for unexpected things to happen during the upgrade because there will be no other programs running.

  13. Mount the required file systems using:

    # fsck -p
# mount -uw /
# mount -at ufs

    Note that the fsck(8) invocation is necessary to update certain fields in the file system superblocks for FreeBSD 5.X. If the system clock tracks local time rather than UTC time, also perform:

    # adjkerntz -i

    During this step, the following warning message may appear on the console (numerous times).

    WARNING: userland calling deprecated sysctl, please rebuild world

    This message can be safely ignored.

  14. Delete the C++ header files with:

    # rm -rf /usr/include/g++

    This step keeps future compilations from accidentally picking up old header files from the GCC 2.95 C++ compiler.

  15. Install the new userland utilities with:

    # cd /usr/src
# make installworld

  16. On systems running named(8), its configuration files need to be moved into a chroot(8) area in /var/named. If any files exist in /var/named, they should be backed up at this point.

    # cd /etc
# mv named named.bak
# mkdir -p /var/named/etc/namedb
# cp -Rp named.bak/* /var/named/etc/namedb

    If the configuration uses the generated localhost files, the following steps may be needed to regenerate them:

    # cd /var/named/etc/namedb
# /bin/sh make-localhost
# rm -f localhost-v6.rev localhost.rev

    Finally, two obsolete binaries from the prior BIND need to be removed:

    # rm /usr/sbin/nslookup
# rm /usr/sbin/nsupdate

  17. Update the system configuration files by running:

    # mergemaster -i

    This step is non-optional. It is required to make the startup and configuration files in /etc consistent with the new kernel and world.

    After invoking mergemaster(8), it is a good idea to inspect (and possibly modify) /etc/rc.conf. A number of default values for the variables defined in this file have changed; in particular, some services that were enabled by default in 4.X are now disabled by default in 5.X.

  18. Remove leftover BIND 8 files:

    # rm /usr/bin/dnskeygen /usr/bin/dnsquery
# rm /usr/libexec/named-xfer
# rm /usr/sbin/named.restart /usr/sbin/ndc

  19. Reboot.

After upgrading the base system, upgrades to some non-base-system components are generally needed to restore normal functionality. Perl is no longer a part of the base system and should be installed from the Ports Collection (specifically, the lang/perl5.8 port) or from a package. After this installation, all ports and/or packages depending on Perl need to be reinstalled.

Running FreeBSD 4.X binaries requires a set of compatability libraries. These are available by installing the misc/compat4x package/port.

As mentioned in a prior section, Xorg is the default implementation of the X Window System. The Ports Collection (as well as packages) rely on this change to satisfy dependencies. To convert the installed windowing system from XFree86 to Xorg, refer to the Installing X11 section in the FreeBSD Handbook.

6 Summary

This article presents some of the more notable new features in FreeBSD 5.X, and lists some areas of particular concern to those users upgrading existing FreeBSD 4.X systems. It also presents two sets of upgrade instructions, one using binaries from installation media and one based on recompiling the base system from source code.

Notes

[1]

Note that execution of a.out executables on the i386 and pc98 is still supported with the COMPAT_AOUT kernel option. Some of the compilers in the Ports Collection (for example, lang/gcc28) are still capable of generating a.out format executables.

This file, and other release-related documents, can be downloaded from ftp://ftp.FreeBSD.org/.

For questions about FreeBSD, read the documentation before contacting <questions@FreeBSD.org>.

For questions about this documentation, e-mail <doc@FreeBSD.org>.