Chapter 8 Disks, File Systems, and Boot Loaders

8.1. How can I add my new hard disk to my FreeBSD system?
8.2. How do I move my system over to my huge new disk?
8.3. Which partitions can safely use Soft Updates? I have heard that Soft Updates on / can cause problems. What about Journaled Soft Updates?
8.4. What is inappropriate about my ccd(4)?
8.5. Why can I not edit the disk label on my ccd(4)?
8.6. Can I mount other foreign file systems under FreeBSD?
8.7. How do I mount a secondary DOS partition?
8.8. Is there a cryptographic file system for FreeBSD?
8.9. How can I use the Windows NT® loader to boot FreeBSD?
8.10. How do I boot FreeBSD and Linux® from LILO?
8.11. How do I boot FreeBSD and Linux using GRUB?
8.12. How do I boot FreeBSD and Linux using BootEasy?
8.13. How do I change the boot prompt from ??? to something more meaningful?
8.14. I have a new removable drive, how do I use it?
8.15. Why do I get “Incorrect super block” when mounting a CD-ROM?
8.16. Why do I get “Device not configured” when mounting a CD-ROM?
8.17. Why do all non-English characters in filenames show up as “?” on my CDs when mounted in FreeBSD?
8.18. I burned a CD under FreeBSD and now I can not read it under any other operating system. Why?
8.19. How can I create an image of a data CD?
8.20. Why can I not mount an audio CD?
8.21. How do I mount a multi-session CD?
8.22. How do I let ordinary users mount CD-ROMs, DVDs, USB drives, and other removable media?
8.23. The du and df commands show different amounts of disk space available. What is going on?
8.24. How can I add more swap space?
8.25. Why does FreeBSD see my disk as smaller than the manufacturer says it is?
8.26. How is it possible for a partition to be more than 100% full?

8.1. How can I add my new hard disk to my FreeBSD system?

See the Adding Disks section in the FreeBSD Handbook.

8.2. How do I move my system over to my huge new disk?

The best way is to reinstall the OS on the new disk, then move the user data over. This is highly recommended if you have been tracking -STABLE for more than one release, or have updated a release instead of installing a new one. You can install booteasy on both disks with boot0cfg(8), and dual boot them until you are happy with the new configuration. Skip the next paragraph to find out how to move the data after doing this.

Should you decide not to do a fresh install, you need to partition and label the new disk with either sysinstall(8), or fdisk(8) and disklabel(8). You should also install booteasy on both disks with boot0cfg(8), so that you can dual boot to the old or new system after the copying is done.

Now you have the new disk set up, and are ready to move the data. Unfortunately, you cannot just blindly copy the data. Things like device files (in /dev), flags, and links tend to screw that up. You need to use tools that understand these things, which means dump(8). Although it is suggested that you move the data in single user mode, it is not required.

You should never use anything but dump(8) and restore(8) to move the root file system. The tar(1) command may work — then again, it may not. You should also use dump(8) and restore(8) if you are moving a single partition to another empty partition. The sequence of steps to use dump to move a partitions data to a new partition is:

  1. newfs the new partition.

  2. mount it on a temporary mount point.

  3. cd to that directory.

  4. dump the old partition, piping output to the new one.

For example, if you are going to move root to /dev/ad1s1a, with /mnt as the temporary mount point, it is:

# newfs /dev/ad1s1a
# mount /dev/ad1s1a /mnt
# cd /mnt
# dump 0af - / | restore rf -

Rearranging your partitions with dump takes a bit more work. To merge a partition like /var into its parent, create the new partition large enough for both, move the parent partition as described above, then move the child partition into the empty directory that the first move created:

# newfs /dev/ad1s1a
# mount /dev/ad1s1a /mnt
# cd /mnt
# dump 0af - / | restore rf -
# cd var
# dump 0af - /var | restore rf -

To split a directory from its parent, say putting /var on its own partition when it was not before, create both partitions, then mount the child partition on the appropriate directory in the temporary mount point, then move the old single partition:

# newfs /dev/ad1s1a
# newfs /dev/ad1s1d
# mount /dev/ad1s1a /mnt
# mkdir /mnt/var
# mount /dev/ad1s1d /mnt/var
# cd /mnt
# dump 0af - / | restore rf -

You might prefer cpio(1), pax(1), tar(1) to dump(8) for user data. At the time of this writing, these are known to lose file flag information, so use them with caution.

8.3. Which partitions can safely use Soft Updates? I have heard that Soft Updates on / can cause problems. What about Journaled Soft Updates?

Short answer: you can usually use Soft Updates safely on all partitions.

Long answer: Soft Updates has two characteristics that may be undesirable on certain paritions. First, a Soft Updates partition has a small chance of losing data during a system crash. (The partition will not be corrupted; the data will simply be lost.) Second, Soft Updates can cause temporary space shortages.

When using Soft Updates, the kernel can take up to thirty seconds to write changes to the physical disk. When a large file is deleted the file still resides on disk until the kernel actually performs the deletion. This can cause a very simple race condition. Suppose you delete one large file and immediately create another large file. The first large file is not yet actually removed from the physical disk, so the disk might not have enough room for the second large file. You get an error that the partition does not have enough space, although you know perfectly well that you just released a large chunk of space! When you try again mere seconds later, the file creation works as you expect. This has left more than one user scratching his head and doubting his sanity, the FreeBSD file system, or both.

If a system should crash after the kernel accepts a chunk of data for writing to disk, but before that data is actually written out, data could be lost. This risk is extremely small, but generally manageable.

These issues affect all partitions using Soft Updates. So, what does this mean for the root partition?

Vital information on the root partition changes very rarely. If the system crashed during the thirty-second window after such a change is made, it is possible that data could be lost. This risk is negligible for most applications, but you should be aware that it exists. If your system cannot tolerate this much risk, do not use Soft Updates on the root file system!

/ is traditionally one of the smallest partitions. If you put the /tmp directory on / and you have a busy /tmp, you might see intermittent space problems. Symlinking /tmp to /var/tmp will solve this problem.

Finally, dump(8) does not work in live mode (-L) on a filesystem, with Journaled Soft Updates (SU+J).

8.4. What is inappropriate about my ccd(4)?

The symptom of this is:

# ccdconfig -C
ccdconfig: ioctl (CCDIOCSET): /dev/ccd0c: Inappropriate file type or format

This usually happens when you are trying to concatenate the c partitions, which default to type unused. The ccd(4) driver requires the underlying partition type to be FS_BSDFFS. Edit the disk label of the disks you are trying to concatenate and change the types of partitions to 4.2BSD.

8.5. Why can I not edit the disk label on my ccd(4)?

The symptom of this is:

# disklabel ccd0
(it prints something sensible here, so let us try to edit it)
# disklabel -e ccd0
(edit, save, quit)
disklabel: ioctl DIOCWDINFO: No disk label on disk;
use "disklabel -r" to install initial label

This is because the disk label returned by ccd(4) is actually a “fake” one that is not really on the disk. You can solve this problem by writing it back explicitly, as in:

# disklabel ccd0 > /tmp/disklabel.tmp
# disklabel -Rr ccd0 /tmp/disklabel.tmp
# disklabel -e ccd0
(this will work now)

8.6. Can I mount other foreign file systems under FreeBSD?

FreeBSD supports a variety of other file systems.

UFS

UFS CD-ROMs can be mounted directly on FreeBSD. Mounting disk partitions from Digital UNIX and other systems that support UFS may be more complex, depending on the details of the disk partitioning for the operating system in question.

ext2/ext3

FreeBSD supports ext2fs and ext3fs partitions. See mount_ext2fs(8) for more information.

NTFS

FUSE based NTFS support is available as a port (sysutils/fusefs-ntfs). For more information see ntfs-3g.

FAT

FreeBSD includes a read-write FAT driver. For more information, see mount_msdosfs(8).

ZFS

FreeBSD includes a port of Sun™'s ZFS driver. The current recommendation is to use it only on amd64 platforms with sufficient memory. For more information, see zfs(8).

FreeBSD also supports network file systems such as NFS (see mount_nfs(8)), NetWare (see mount_nwfs(8)), and Microsoft-style SMB file systems (see mount_smbfs(8)). You can find ports based on FUSE (sysutils/fusefs-kmod) for many other file systems.

8.7. How do I mount a secondary DOS partition?

The secondary DOS partitions are found after all the primary partitions. For example, if you have an “E” partition as the second DOS partition on the second SCSI drive, there will be a device file for “slice 5” in /dev, so simply mount it:

# mount -t msdosfs /dev/da1s5 /dos/e

8.8. Is there a cryptographic file system for FreeBSD?

Yes. You can use either gbde(8) or geli(8), see the Encrypting Disk Partitions section of the FreeBSD Handbook.

8.9. How can I use the Windows NT® loader to boot FreeBSD?

The general idea is that you copy the first sector of your native root FreeBSD partition into a file in the DOS/Windows NT partition. Assuming you name that file something like c:\bootsect.bsd (inspired by c:\bootsect.dos), you can then edit c:\boot.ini to come up with something like this:

[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT"
C:\BOOTSECT.BSD="FreeBSD"
C:\="DOS"

If FreeBSD is installed on the same disk as the Windows NT boot partition simply copy /boot/boot1 to C:\BOOTSECT.BSD. However, if FreeBSD is installed on a different disk /boot/boot1 will not work, /boot/boot0 is needed.

/boot/boot0 needs to be installed using sysinstall(8) by selecting the FreeBSD boot manager on the screen which asks if you wish to use a boot manager. This is because /boot/boot0 has the partition table area filled with NULL characters but sysinstall(8) copies the partition table before copying /boot/boot0 to the MBR.

Warning: Do not simply copy /boot/boot0 instead of /boot/boot1; you will overwrite your partition table and render your computer un-bootable!

When the FreeBSD boot manager runs it records the last OS booted by setting the active flag on the partition table entry for that OS and then writes the whole 512-bytes of itself back to the MBR so if you just copy /boot/boot0 to C:\BOOTSECT.BSD then it writes an empty partition table, with the active flag set on one entry, to the MBR.

8.10. How do I boot FreeBSD and Linux® from LILO?

If you have FreeBSD and Linux on the same disk, just follow LILO's installation instructions for booting a non-Linux operating system. Very briefly, these are:

Boot Linux, and add the following lines to /etc/lilo.conf:

other=/dev/hda2
        table=/dev/hda
        label=FreeBSD

(the above assumes that your FreeBSD slice is known to Linux as /dev/hda2; tailor to suit your setup). Then, run lilo as root and you should be done.

If FreeBSD resides on another disk, you need to add loader=/boot/chain.b to the LILO entry. For example:

other=/dev/dab4
        table=/dev/dab
        loader=/boot/chain.b
        label=FreeBSD

In some cases you may need to specify the BIOS drive number to the FreeBSD boot loader to successfully boot off the second disk. For example, if your FreeBSD SCSI disk is probed by BIOS as BIOS disk 1, at the FreeBSD boot loader prompt you need to specify:

Boot: 1:da(0,a)/boot/kernel/kernel

You can configure boot(8) to automatically do this for you at boot time.

The Linux+FreeBSD mini-HOWTO is a good reference for FreeBSD and Linux interoperability issues.

8.11. How do I boot FreeBSD and Linux using GRUB?

Booting FreeBSD using GRUB is very simple. Just add the following to your configuration file /boot/grub/menu.lst (or /boot/grub/grub.conf in some systems, e.g., Red Hat Linux and its derivatives).

title FreeBSD 6.1
	root (hd0,a)
	kernel /boot/loader

Where hd0,a points to your root partition on the first disk. If you need to specify which slice number should be used, use something like this (hd0,2,a). By default, if the slice number is omitted, GRUB searches the first slice which has a partition.

8.12. How do I boot FreeBSD and Linux using BootEasy?

Install LILO at the start of your Linux boot partition instead of in the Master Boot Record. You can then boot LILO from BootEasy.

If you are running Windows® and Linux this is recommended anyway, to make it simpler to get Linux booting again if you should need to reinstall Windows (which is a Jealous Operating System, and will bear no other Operating Systems in the Master Boot Record).

8.13. How do I change the boot prompt from ??? to something more meaningful?

You can not do that with the standard boot manager without rewriting it. There are a number of other boot managers in the sysutils ports category that provide this functionality.

8.14. I have a new removable drive, how do I use it?

If the drive already has a file system on it, you can use a command like this:

# mount -t msdosfs /dev/da0s1 /mnt

If the drive will only be used with FreeBSD systems it is better idea to stick a BSD file system on it, like UFS or ZFS. You will get long filename support, at least a 2X improvement in performance, and a lot more stability. If the drive will be used by other operating systems a more portable choice, such as msdosfs, is better.

# dd if=/dev/zero of=/dev/da0 count=2
# gpart create -s GPT /dev/da0
# gpart add -t freebsd-ufs /dev/da0

Finally, create a new file system:

# newfs /dev/da0p1

and mount it:

# mount /dev/da0s1 /mnt

It is a good idea to add a line to /etc/fstab (see fstab(5)) so you can just type mount /mnt in the future:

/dev/da0p1 /mnt ufs rw,noauto 0 0

8.15. Why do I get “Incorrect super block” when mounting a CD-ROM?

You have to tell mount(8) the type of the device that you want to mount. This is described in the Handbook section on optical media, specifically the section Using Data CDs.

8.16. Why do I get “Device not configured” when mounting a CD-ROM?

This generally means that there is no CD-ROM in the CD-ROM drive, or the drive is not visible on the bus. Please see the Using Data CDs section of the Handbook for a detailed discussion of this issue.

8.17. Why do all non-English characters in filenames show up as “?” on my CDs when mounted in FreeBSD?

Your CD-ROM probably uses the “Joliet” extension for storing information about files and directories. This is discussed in the Handbook chapter on creating and using CD-ROMs, specifically the section on Using Data CD-ROMs.

8.18. I burned a CD under FreeBSD and now I can not read it under any other operating system. Why?

You most likely burned a raw file to your CD, rather than creating an ISO 9660 file system. Take a look at the Handbook chapter on creating CD-ROMs, particularly the section on burning raw data CDs.

8.19. How can I create an image of a data CD?

This is discussed in the Handbook section on duplicating data CDs. For more on working with CD-ROMs, see the Creating CDs Section in the Storage chapter in the Handbook.

8.20. Why can I not mount an audio CD?

If you try to mount an audio CD, you will get an error like “cd9660: /dev/acd0c: Invalid argument”. This is because mount only works on file systems. Audio CDs do not have file systems; they just have data. You need a program that reads audio CDs, such as the audio/xmcd port.

8.21. How do I mount a multi-session CD?

By default, mount(8) will attempt to mount the last data track (session) of a CD. If you would like to load an earlier session, you must use the -s command line argument. Please see mount_cd9660(8) for specific examples.

8.22. How do I let ordinary users mount CD-ROMs, DVDs, USB drives, and other removable media?

As root set the sysctl variable vfs.usermount to 1.

# sysctl vfs.usermount=1

To make this persist across reboots, add the line vfs.usermount=1 to /etc/sysctl.conf so that it is reset at system boot time.

Users can only mount devices they have read permissions to. To allow users to mount a device permissions must be set in /etc/devfs.conf.

For example, to allow users to mount the first USB drive add:

# Allow all users to mount a USB drive.
	    own       /dev/da0       root:operator
	    perm      /dev/da00       0666

All users can now mount devices they could read onto a directory that they own:

% mkdir ~/my-mount-point
% mount -t msdosfs /dev/da0~/my-mount-point

Unmounting the device is simple:

% umount ~/my-mount-point

Enabling vfs.usermount, however, has negative security implications. A better way to access MS-DOS® formatted media is to use the emulators/mtools package in the Ports Collection.

Note: The device name used in the previous examples must be changed according to your configuration.

8.23. The du and df commands show different amounts of disk space available. What is going on?

You need to understand what du and df really do. du goes through the directory tree, measures how large each file is, and presents the totals. df just asks the file system how much space it has left. They seem to be the same thing, but a file without a directory entry will affect df but not du.

When a program is using a file, and you delete the file, the file is not really removed from the file system until the program stops using it. The file is immediately deleted from the directory listing, however. You can see this easily enough with a program such as more. Assume you have a file large enough that its presence affects the output of du and df. (Since disks can be so large today, this might be a very large file!) If you delete this file while using more on it, more does not immediately choke and complain that it cannot view the file. The entry is simply removed from the directory so no other program or user can access it. du shows that it is gone — it has walked the directory tree and the file is not listed. df shows that it is still there, as the file system knows that more is still using that space. Once you end the more session, du and df will agree.

Note that Soft Updates can delay the freeing of disk space; you might need to wait up to 30 seconds for the change to be visible!

This situation is common on web servers. Many people set up a FreeBSD web server and forget to rotate the log files. The access log fills up /var. The new administrator deletes the file, but the system still complains that the partition is full. Stopping and restarting the web server program would free the file, allowing the system to release the disk space. To prevent this from happening, set up newsyslog(8).

8.24. How can I add more swap space?

In the Configuration and Tuning section of the Handbook, you will find a section describing how to do this.

8.25. Why does FreeBSD see my disk as smaller than the manufacturer says it is?

Disk manufacturers calculate gigabytes as a billion bytes each, whereas FreeBSD calculates them as 1,073,741,824 bytes each. This explains why, for example, FreeBSD's boot messages will report a disk that supposedly has 80 GB as holding 76,319 MB.

Also note that FreeBSD will (by default) reserve 8% of the disk space.

8.26. How is it possible for a partition to be more than 100% full?

A portion of each UFS partition (8%, by default) is reserved for use by the operating system and the root user. df(1) does not count that space when calculating the Capacity column, so it can exceed 100%. Also, you will notice that the Blocks column is always greater than the sum of the Used and Avail columns, usually by a factor of 8%.

For more details, look up -m in tunefs(8).

8.1 ZFS

8.1.1. What is the minimum amount of RAM one should have to run ZFS?
8.1.2. What is the ZIL and when does it get used?
8.1.3. Do I need a SSD for ZIL?
8.1.4. What is the L2ARC?
8.1.5. Is enabling deduplication advisable?
8.1.6. I can not delete or create files on my ZFS pool. How can I fix this?
8.1.7. Does ZFS support TRIM for Solid State Drives?

8.1.1. What is the minimum amount of RAM one should have to run ZFS?

A minimum of 4GB of RAM is required for comfortable usage, but individual workloads can vary widely.

8.1.2. What is the ZIL and when does it get used?

The ZIL ((ZFS intent log) is a write log used to implement posix write commitment semantics across crashes. Normally writes are bundled up into transaction groups and written to disk when filled (“Transaction Group Commit”). However syscalls like fsync(2) require a commitment that the data is written to stable storage before returning. The ZIL is needed for writes that have been acknowledged as written but which are not yet on disk as part of a transaction. The transaction groups are timestamped. In the event of a crash the last valid timestamp is found and missing data is merged in from the ZIL.

8.1.3. Do I need a SSD for ZIL?

By default, ZFS stores the ZIL in the pool with all the data. If your application has a heavy write load, storing the ZIL in a separate device that has very fast synchronous, sequential write performance can improve overall system. For other workloads, a SSD is unlikely to make much of an improvement.

8.1.4. What is the L2ARC?

The L2ARC is a read cache stored on a fast device such as an SSD. This cache is not persisent across reboots. Note that RAM is used as the first layer of cache and the L2ARC is only needed if there is insufficient RAM.

L2ARC needs space in the ARC to index it. So, perversely, a working set that fits perfectly in the ARC will not fit perfectly any more if a L2ARC is used because part of the ARC is holding the L2ARC index, pushing part of the working set into the L2ARC which is slower than RAM.

8.1.5. Is enabling deduplication advisable?

Generally speaking, no.

Deduplication takes up a significant amount of RAM and may slow down read and write disk access times. Unless one is storing data that is very heavily duplicated (such as virtual machine images, or user backups) it is possible that deduplication will do more harm than good. Another consideration is the inability to revert deduplication status. If data is written when deduplication is enabled, disabling dedup will not cause those blocks which were deduplicated to be replicated until they are next modified.

Deduplication can also lead to some unexpected situations. In particular deleting files may become much slower.

8.1.6. I can not delete or create files on my ZFS pool. How can I fix this?

This could happen because the pool is 100% full. ZFS requires space on the disk in order to write transaction metadata. In order to restore the pool to a usable state, truncate a file you want to delete.

% truncate -s 0 unimportant-file

File truncation works because a new transaction is not started, new spare blocks are created instead.

Note: On systems with additional ZFS dataset tuning, such as deduplication, the space may not be immediately available

8.1.7. Does ZFS support TRIM for Solid State Drives?

ZFS TRIM support was added to FreeBSD 10-CURRENT with revision r240868. ZFS TRIM support is not yet available on the -STABLE branches.

ZFS TRIM is enabled by default, and can be turned off by adding this line to /etc/sysctl.conf:

vfs.zfs.trim_disable=1

Note: ZFS TRIM may not work with all configurations, such as a ZFS filesystem on a GELI-backed device.