Overview
The purpose of this document is to keep track of the steps I performed in creating and booting a Debian install within a directory of an existing Linux system for the purpose of exporting as an NFS root. These instructions are written with a diskless computer booting via PXE and configured by DHCP in mind. If you plan to use a different system, replace programs like dhcp3-server and pxelinux as needed. Additionally, I am adding notes about things that, in hindsight, may have been easier.
Why?
I created this NFS root for booting a Soekris Embedded Device. (I used a net4801.) These instructions are based on the guide Mike Machado created Here, which was written for Debian Potato. Before continuing with these instructions, consider your own "why?", since the answers may change how you proceed.
Terminology
A brief note on the language I'm using. The NFS root export directory will be called the "root filesystem area". The distribution installed there will be called the "installation". The original distribution running on this computer will be called the "host", and the computer will be the "host machine". The computer that will be using the NFS share will be called the "remote machine".
Preparation
Following the instructions and examples on Mike Machado's site, I installed the dhcp3-server and the tftpd-hpa package. I already had the NFS server and its associated dependencies installed, as well as the syslinux package which provides the pxelinux image. Following his directions, the next thing to do would have been to get the root image used by Potato. At one time, Debian's base installation included a tarball that had an entire installed and configured root filesystem in it. Starting with Debian Woody (3.0), this has been replaced by a tarball containing the base debian packages, so that they can be installed and uninstalled "properly" by the installer.
Before you continue, I strongly suggest you turn off all servers you aren't using. The installation script for many of the daemons will attempt to restart the server if its running, using the version in the root filesystem area. Also, in order for cron to install successfully, you'll have to mount proc within the root filesystem area, with mount -t proc none /nfs/root/proc. As an extra note, be careful with installing lilo.
Extracting the root filesystem
Note: This is written assuming that your host system is a Debian ssystem, so that dpkg is available. If not, you can still do this, but you'll need to extract everything first. To extract a .deb file, use ar x file.deb. This will extract the control and contents as a tarball, which you can then extract with tar.
Since the root filesystem is no longer a nice, simple tarball, we have to extract and install the packages ourselves. This isn't as easy a task as dpkg's manpage makes it sound, as the --root= flag expects the given directory to already include a working debian installation. To begin, you need to have a directory to install into. I am using /nfs/root/ for the root directory. Then, extract the basedebs.tar file you can obtain from Debian's base-images directory. This will create a var/apt/cache/archives/ directory containing the base .debs that Woody uses. Depending on how you intend to use this system, you can remove some of the debs such as the packages for ppp and pppoe.
From here, I proceeded the hard way, by individually extracting debs with dpkg -x filename.deb /nfs/root/ one at a time and testing whether I could successfully install the package using dpkg --root=/nfs/root -i filename.deb. This took a while, but allowed me to watch what each package contained, and to make sure each were done in the proper order. I also had to "cheat" on some of the circular dependencies by making /nfs/root/var/lib/dpkg/status show one package as properly installed so I could install the other, then setting it back to install the first one properly. This was only a problem on things like glibc.
Since then, I've decided that a faster way to do this would have been to extract all of the debs at once, then use chroot /nfs/root /bin/bash then dpkg -i /var/cache/apt/archives/*.deb to allow dpkg to attempt to sort out the circular dependencies itself.
Configuring and Finishing the installation
Before you begin the configuration step, you'll need to mount the /proc directory into your NFS area. Some of the configuration scripts require various information out of /proc (keep in mind that this information will be WRONG with respect to the remote machine, and you may need to reconfigure some packages later). You can do this by running mount -t proc proc /nfs/root/proc/ on the host machine before chrooting to /nfs/root/ for the remainder of the process.
You'll need to configure the new installation as you go. Exim will not start without being able to find its hostname, and the package installation will fail until exim can be started. To fix this, copy the /etc/hosts and /etc/resolv.conf file from the host into the installation and configure exim again. Around this point you'll need to decide what you're going to be doing with this system. If this is going to be deployed to several computers, you'll probably want to make the configuration as generic as possible (for example, listing all of the hostnames in /etc/hosts).
Once everything installs and configures successfully, edit /nfs/root/etc/apt/sources.list and make sure it points to a stable mirror you like. chroot into the installation, and use apt-get to upgrade to the current versions of the base packages before you install any new packages. Be sure to use chroot so you don't mess up the host! If you wish to change to Sarge (testing) or Sid (unstable), update sources.list now, and redo the upgrade. Once you have the base packages up-to-date for the version of Debian you want to use, then you can start installing whatever other packages you wish to use.
The last part is to set up the mounted filesystems in /nfs/root/etc/fstab. Depending on the number of systems that will be using this installation at the same time, you can either mount everything from NFS, or you can have locally mounted ramdisks for directories like /var/spool and /var/log. The latter is preferred if you're going to have more than one system use the installation at a time. Either way, you'll need to specify the root filesystem like so:
host:/nfs/root / nfs defaults 0 0
Where host is the IP address or a resolvable hostname of the host machine. If you do not include this line, the kernel's NFS mounted root will still be there, however it will not automatically be remounted read-write, and many programs may not work. If you intend the nfs mount to remain read-only so that it can be used by more than one installation, change defaults to defaults,ro.
Configuring a Kernel
While Debian Woody will support any version of the kernels, if you plan on using a 2.6 series kernel you will want to upgrade to Sarge for the full effect. The main Sarge package repository has programs that Woody doesn't, like ebtables that use 2.6's new features. Your other choice is to use an alternative apt repository like apt-get.org or backports.org to get versions of these packages that will fit in with Woody.
When compiling the kernel, you'll need to remember that you're building it for the client machine, not the host machine. Be sure to include the drivers your intended client machine will be using. Additionally you'll need the following items (the locations given are for menuconfig's organization in 2.6.7):
Once you have selected these options, go back and doublecheck the options you need specific to your installation. Then build the kernel and kernel modules. If you need to create an initrd for booting with certain modules, create this image now. The modules will need to be installed into the appropriate directory on the installation root, so that you can load them later. Put the kernel into the tftp server root directory (see below).
Preparing to Boot
You should now have a complete installation ready to go. Now you need to prepare your host computer for hosting a remote boot. The first step is to set up the pxelinux image which will fetch and boot a linux kernel over the network. Create a directory for the root of the tftp server (traditionally /tftpboot) and copy the pxelinux image from the syslinux package into it (traditionally named pxelinux.0). Then, create the /tftpboot/pxelinux.cfg/ directory, where the kernel configurations will reside. The next step will depend on how many systems you are going to use this image on. If you are only going to boot a single installation, then use "default" as the filename for the configuration. Otherwise, see the pxelinux.doc file in the syslinux documentation directory for instructions on the naming scheme used by pxelinux.
Within the configuration file, you will need to specify the name of the kernel image stored in the /tftpboot/ directory. For the options, you will need root=/dev/nfs nfsroot=hostIP:/nfs/root ip=<clientIP>:<serverIP>:<gatewayIP>:<netmask>:<hostname>:<device>:<autoconf>, in addition to whatever other options you'll need for the kernel. The root=/dev/nfs option tells the kernel that the root filesystem will be mounted via NFS. The nfsroot=hostIP:/nfs/root option tells the kernel the IP address of the NFS server with the root filesystem, and the directory to mount. The third option, ip=... (note: on older kernels this used to be called nfsaddrs=...) is mostly optional when using kernel level autoconfiguration. In that case, the kernel can get its configuration from the dhcp server and you just need to specify ip=DHCP to tell the kernel that all of the values are obtained by DHCP. Otherwise, you can specify the networking information for the installation by hand. The serverIP parameter here is overloaded to mean several things. If you are using the autoconfiguration, then it is the IP address of the DHCP server. Otherwise it should be blank or set to the IP of the NFS server. The device parameter specifies which device should be used for the DHCP request. In all, my kernel configuration (showing manual ip configuration) is:
SERIAL 0 19200
DEFAULT vmlinuz console=ttyS0,19200n81 root=/dev/nfs nfsaddrs=192.168.0.5:192.168.0.1:192.168.0.1:255.255.255.0:soekris:eth0 nfsroot=192.168.0.1:/nfs/root panic=10
IPAPPEND 1
This includes extra configuration needed for using the Soekris device's serial port for a serial console.
(I was having trouble with ip=DHCP in 2.6.7, it seems that Linux tries to renew a lease from the wrong IP address, and if you don't have your DHCP server set as "authoritative" then the server will not reject the IP, but won't grant it either, so the kernel loops forever trying to get the wrong IP. Be sure that your config file has the line authoritative; ABOVE the subnet declaration.)
Next, you need to set up the tftp server itself. This should be set up on the same host with the NFS server and the DHCP server for ease of configuration. Debian Sid's tftpd-hpa package will automatically create a commented out line in your /etc/inetd.conf file, which you will need to adjust. As of package version 0.36-1, the server is incorrectly configured to run as nobody, it must run as root. Also, change the parameters to include the "-s" flag. This will cause the server to chroot into its root directory specified at the end of the line (/tftpboot).
Then, configure the NFS server. Add the line /nfs/root installationIP(rw,no_root_squash) to /etc/exports and check that /etc/hosts.allow and /etc/hosts.deny will not block your installation from accessing the NFS server or the portmapper. (If you have problems later with mounting the root, check /var/log/daemon.log, which will have messages from both the portmapper and NFS about any access denied errors. If no errors appear, check that you have set the correct IP addresses in the kernel configuration, and that the firewall is not preventing the installation from mounting the NFS share.) If you have more than one installation accessing this, you can use wildcards or a network/netmask pair in place of installationIP.
Finally, configure and start the dhcp server. The dhcp server will not start at all with the configuration shipped by debian. You will need to specify a subnet and netmask for the server to issue IP addresses from (I used subnet 192.168.0.0 netmask 255.255.255.0 { }) and then configure entries for the installation(s) to use. In my case, I used the ethernet hardware/MAC address to specify a static IP allocation:
host soekris {
hardware ethernet 00:00:00:00:00:00;
fixed-address 192.168.0.5;
option subnet-mask 255.255.255.0;
option broadcast-address 192.168.0.255;
option routers 192.168.0.1;
filename "pxelinux.0";
}
You will need to adjust these values to fit your needs.
Running Debian
From this point on, Debian should operate normally from the remote system over NFS. You can reduce your bandwidth needs if you create local mounts (either on a local drive or using tmpfs) for filesystems like /tmp and /var/log/. It is highly suggested that once you reach this point, that you use the remote system to perform any dpkg/apt-get operations. This will prevent confusion with other services already running on the host (especially when upgrading init), which may occur even when using --root with dpkg or chroot.
This document was last updated 2005-02-12