AUTHOR: Matthias S. Benkmann
DATE: 2004-12-24 LICENSE: Creative Commons Attribution-NoDerivs 2.0 (http://creativecommons.org/licenses/by-nd/2.0/) SYNOPSIS: Install LFS next to an existing OS, not on a fresh partition (v1.0) DESCRIPTION: This hint presents a simple and safe way to install one or more LFS systems on a pre-existing partition that may already contain data from other (even non-Linux) operating systems. The different systems will be bootable independent from each other and run with no loss of performance. If you want to build an LFS system but can not or do not want to give it its own dedicated partition, this hint is for you. ATTACHMENTS: http://www.linuxfromscratch.org/hints/downloads/attachments/lfs_next_to_existing_systems/lfs_next_helpers.tar.bz2 PREREQUISITES: None. HINT: ########################################################################### Contents ########################################################################### 1. The LFS Partition 1.1 Choosing the LFS Partition 1.2 Creating the LINUX_FROM_SCRATCH Directory 1.3 Matching Up Hint and Book 2. Building the LFS System 3. Making the LFS System Bootable 3.1 First Read Then Proceed 3.2 The Problem 3.3 The Solution: pre-init 3.4 The Boot Manager is Special 3.5 Making the LFS System Bootable 3.6 /dev/console 4. Accessing the Full LFS Partition 5. Elevating the LFS system to / 5.1 Introduction 5.2 Don't Saw the Branch You're Sitting On 5.3 Don't Break Your Boot Manager 5.4 Don't Touch Special Directories and Files A. Acknowledgements and Changelog ########################################################################### Before You Begin ########################################################################### Just to avoid misunderstandings let me make clear that this hint is *not* about cross-compiling an LFS system from a non-Linux operating system. This hint addresses only the aspects relevant to building and booting an LFS system next to other data on an existing partition rather than using a fresh, empty, dedicated partition. This hint is mostly independent of the actual instructions used to build the LFS system, so you may use this hint in conjuction with a cross-compiling hint. However, this hint is written under the assumption that you are following the LFS book, i.e. you're using a Linux system as host for the build, and any shell commands given in this hint are meant to be executed on a Linux host. ########################################################################### 1. The LFS Partition ########################################################################### 1.1 Choosing the LFS Partition ------------------------------ The LFS book explains how to create a new partition for your LFS system and how to set it up properly. When you use this hint you don't need to do this. Instead you pick one of your existing partitions to install on. There are some conditions that a partition needs to fulfill in order to be usable with this hint. If your chosen partition belongs to a Linux system (e.g. your host distro), condition 1 is probably the only thing you need to worry about. But if you want to install to a partition from a non-Linux OS, read them all carefully. 1) The partition needs to have enough free space. This restriction should be obvious. See the LFS book for an estimate of how much space you are going to need. 2) The filesystem used on the partition must offer the usual Linux/Unix features such as user/group/other permissions, hard and symbolic links. This hint does not use loopback-mounted files or other tricks to install one filesystem on another filesystem. Therefore the target partition's filesystem needs to support the features necessary for a Linux installation directly. And of course these features need to be supported by the Linux driver for that filesystem. Unfortunately this rules out VFAT and NTFS, so installing LFS on a Windows partition is not possible with this hint. But partitions from other *nixes may work (e.g. the HFS+ filesystem from Mac OS X has been confirmed to support all features, including device nodes). If you are unsure about your chosen partition, contact me or ask on the lfs-support mailing list. Note that Linux device nodes don't need to be supported on the target partition. A workaround for partitions that don't support Linux device nodes is offered later in this hint. 3) The partition needs to be usable as a root partition. This restriction is usually a non-issue. It just means that the kernel needs to be able to use the chosen partition as root filesystem. Partitions that satisfy condition 2 should also satisfy this condition, but there may be some filesystem that the kernel can't use as root fs. 4) If there is a file /dev/console on the partition, attempting to open it must be safe. The Linux kernel attempts to open /dev/console on the root partition during the boot. If this file does not exist, is a regular file or an actual Linux device node for the console this is harmless, but if it is something else (e.g. a device node for a non-Linux OS), you should make sure that the kernel's attempt to open it is safe. Contact me or ask on lfs-support if you do indeed have a weird /dev/console on your target partition and are unsure about its safety. BTW, it is trivial to patch the Linux kernel to not open /dev/console and this will be described later in this hint, so even if you should have an unsafe /dev/console, this need not be a show-stopper. NOTE: To find out if you do have a /dev/console on your chosen target partition, you need to boot with a system that does not use the /dev directory on that partition (e.g. boot from a rescue disk). Many systems use virtual /dev directories that contain files only while the system is running. These virtual files don't matter, because they won't exist when you boot your LFS system. 1.2 Creating the LINUX_FROM_SCRATCH Directory --------------------------------------------- After you've chosen a target partition, mount it if it isn't already. To keep the files of the new LFS system separate from those of other operating systems, create a new directory LINUX_FROM_SCRATCH somewhere on the target partition. It does not need to be a top-level directory and the name is arbitrary. This hint will refer to this directory as /.../LINUX_FROM_SCRATCH. Substitute the appropriate path to your LFS directory whenever you see this. 1.3 Matching Up Hint and Book ----------------------------- The LFS book is written under the assumption that the new LFS system will be installed to the root directory / of the target partition. But with this hint you will install into the /.../LINUX_FROM_SCRATCH directory instead. Take for instance the following fragment from the LFS book: Throughout this book, the environment variable LFS will be used several times. It is paramount that this variable is always defined. It should be set to the mount point chosen for the LFS partition. Check that the LFS variable is set up properly with: echo $LFS Make sure the output shows the path to the LFS partition's mount point, which is /mnt/lfs if the provided example was followed. If you followed these instructions exactly and set the variable LFS to the mount point of the LFS partition, you'd install to the / directory of the target partition. To match up the LFS book's instructions and this hint, create a directory /mnt/lfs and use the following command: mount --bind /.../LINUX_FROM_SCRATCH /mnt/lfs to do a virtual mount of /.../LINUX_FROM_SCRATCH on /mnt/lfs. After doing this your situation will match that in the LFS book exactly. You have a mount point /mnt/lfs and it's where your LFS system will be built. As a consequence you can set your LFS variable to /mnt/lfs just as done in the standard book instructions. Of course you could just as well skip the mount --bind command and substitute /.../LINUX_FROM_SCRATCH whenever the book talks about "the mount point of the LFS partition". NOTE: When the LFS book does not talk about the mount point, but about the partition itself or its corresponding block device /dev/[xxx], the book's instructions apply unchanged, i.e. "/dev/[xxx]" remains "/dev/[xxx]" and does *not* become "/.../LINUX_FROM_SCRATCH/dev/[xxx]" or whatever. ########################################################################### 2. Building the LFS System ########################################################################### Except for the points discussed in the previous chapter with respect to the LFS partition and its mount point, the LFS book's instructions apply unchanged up to and including the last package of Chapter 6. After you've installed that, extract the lfs_next_helpers.tar.bz2 archive and install it (from inside the chroot environment, just like the other Chapter 6 packages) using the following command: make install This will install the program /sbin/pre-init, which allows booting of a Linux system whose filesystem root is not the root directory of its partition (as is the case when building according to this hint). After installing lfs_next_helpers, continue with the book up to but not including the chapter "Making the LFS System Bootable". ########################################################################### 3. Making the LFS System Bootable ########################################################################### 3.1 First Read Then Proceed --------------------------- If you're not already familiar with it, read the LFS book's chapter "Making the LFS System Bootable" in its entirety now before you do anything else. Then read the rest of this chapter in the hint. After that, follow the instructions from the LFS book, modifying them as appropriate. 3.2 The Problem --------------- What would happen if you attempted to boot your LFS system with the instructions from the book? Let's leave aside the boot manager related problems for the moment. These are relatively minor and will be discussed later. The major problem will manifest itself after the kernel has finished loading and wants to start /sbin/init. init is the parent of all processes, responsible for starting the boot scripts, for instance. The LFS book's instructions build init dynamically linked by default. This means that for loading init the kernel needs the dynamic linker whose path is hardwired in the init binary (e.g. to "/lib/ld-linux.so.2" on the i386-pc-linux-gnu platform). Here's the first problem. The hardwired path starts with "/lib", but our LFS system's lib directory is not /lib but /.../LINUX_FROM_SCRATCH/lib. So unless you have another /lib directory on the target partition (e.g. from your host distro) that contains a compatible dynamic linker, the kernel won't be able to load init. But the problems don't end there. init and the boot scripts want to read their config files from the /etc directory, but the proper directory to read from is /.../LINUX_FROM_SCRATCH/etc. 3.3 The Solution: pre-init -------------------------- The solution to the problem above is similar to the way Chapter 6 is built. Chapter 6 uses the chroot command to create an environment whose filesystem root is /.../LINUX_FROM_SCRATCH. For solving the boot problem the same method is used, but in a more extreme way. The new LFS system shall be self-sufficient, so booting up a host system and then using chroot on that host system to start the LFS system is unsatisfactory. Therefore the chroot command has to be inserted into the boot process, before init is even started. This way init itself will already work with the changed filesystem root and because init is the parent of all processes, all other processes see only the changed root, too. Furthermore LFS's own boot scripts will be responsible for bringing up the system and not the scripts from a host OS. The tool to make all of this happen is the pre-init program which you installed from the lfs_next_helpers archive. pre-init uses the chroot(2) system call (not the /bin/chroot binary) to change the filesystem root, performs some other preparations necessary for a successful boot and then replaces itself with the init from the LFS system, which will start up and work as if it were installed on its own partition. Because pre-init is statically linked and does not use any external binaries it has no dependencies on a host system. 3.4 The Boot Manager is Special ------------------------------- Before you can set up your system to boot via pre-init, it is important that you give special consideration to some aspects of the boot manager. The first thing that you have to consider is that there can be only one master boot manager in the master boot record (MBR) of a hard disk and only one non-master boot manager per partition. The fact that you're using this hint suggests that you have a special interest in keeping your other operating system(s) untouched. Therefore you should think well about the consequences before you make the decision to install a new boot manager. If you already have a boot manager that can boot Linux systems (which is probably the case if you're booting your host system from disk) there is usually no need to install a new one. Don't forget that the boot manager has no influence on the performance (or the "purity") of your LFS system. If your existing boot manager can handle it, the safest and simplest course of action is to use it for booting the new LFS system. The other important detail you have to consider is that the boot manager is loaded before pre-init can change the filesystem root. The boot manager always sees a physical view of the disk and partitions. All paths it uses at run-time will refer to the physical directory structure of the partitions with no consideration of any mounts or changes in filesystem root that may be active once the system is booted. An example should make this clear: Example: Your host system has a separate partition for /usr and you have chosen that partition to install the new LFS system on. You have created the directory /usr/LINUX_FROM_SCRATCH as the root directory for the LFS system. You have chosen to use the mount --bind trick suggested in this hint to make /usr/LINUX_FROM_SCRATCH available as /mnt/lfs under the host system. Viewed from inside Chapter 6's chroot environment, the stage* files of the boot manager GRUB are found in /boot/grub. Viewed from your host system outside chroot, the GRUB files are found in /mnt/lfs/boot/grub as well as /usr/LINUX_FROM_SCRATCH/boot/grub. The boot loader sees the files at (hd?,?)/LINUX_FROM_SCRATCH/boot/grub. This issue is the most likely source of confusion when using this hint. If booting doesn't work as expected, always check your paths. A typical mistake would be to copy a new kernel to the directory /boot from inside the new LFS system, to specify (hd?,?)/boot/ in GRUB's menu.lst and to be confused when GRUB complains that it can't find the kernel image. Another typical mistake would be to add a new entry to /boot/grub/menu.lst inside the new LFS system and to wonder why it doesn't show up in the boot menu. NOTE: If you use LILO instead of GRUB you have to keep in mind that LILO does not load config files at boot time and does not use the directory structure to find the kernel image at boot time. It's all pre-computed at the time you run /sbin/lilo. Therefore, when you use LILO, only the paths that you pass on the kernel command line (e.g. the "init=..." parameter) have to reflect the physical directory structure. The paths that are only read by /sbin/lilo are ordinary user space paths. Disclaimer: I know little about LILO. Take the above with a grain of salt. 3.5 Making the LFS System Bootable ---------------------------------- To make your LFS system bootable follow the instructions in the LFS book (taking the issues mentioned in the previous section into account, especially regarding the paths). When you come to the point of adding an entry for your new LFS system to your boot manager's configuration, there is one small change you have to make. Just add init=/.../LINUX_FROM_SCRATCH/sbin/pre-init to the kernel parameters. Make sure that /.../LINUX_FROM_SCRATCH reflects the *physical* location of the directory on the partition. My entry in GRUB's menu.lst looks like this title LFS system in /lfs on /dev/hda2 root (hd1,1) kernel /boot/lfskernel-x.y.z root=/dev/hda2 init=/lfs/sbin/pre-init Note that my kernel is /boot/lfskernel-x.y.z and not /lfs/boot/lfskernel-x.y.z. /boot is the directory at the top-level of /dev/hda2, not the one in /lfs. I take care to make sure that /boot and /lfs/boot are always in sync. In fact I have set up /boot as a symlink to lfs/boot (Notice that the other way around, i.e. /lfs/boot as symlink to ../boot wouldn't work!). pre-init understands some options that you can pass on the kernel command line: lfsdir= Overrides the directory to use as filesystem root. The default is determined by stripping the suffix "/sbin/..." from the location of the pre-init binary as passed to the kernel with the "init=..." argument. realinit= Overrides the program to execute after pre-init has completed its preparations. The default is "/sbin/init". This option is useful for instance when you want to boot with /bin/sh as init for maintenance work. may be specified with or without a leading prefix. makedev Tells pre-init to mount a ramfs on /dev and to create device nodes for console, null and tty1 on this ramfs. This allows you to boot from a partition that does not support Linux device nodes. Note that this is *not* redundant, even if your boot scripts contain commands to set up /dev. Without a working /dev/console (or alternatively /dev/null) the normal init used by LFS won't even get to the point of starting your boot scripts. pre-init will pass any command line arguments that it doesn't interpret on to the real init. 3.6 /dev/console ---------------- Before the kernel starts pre-init, it tries to open /dev/console as standard input/output/error for it. It does this, because normal programs, including the usual inits (but not pre-init), require that these standard file descriptors are open. If your LFS partition does not have a /dev/console at the top-level (which it probably doesn't if you're not installing to the root partition of another Linux system), then you'll get the following message: Warning: unable to open an initial console. This warning is harmless, because one of the things pre-init does when preparing to boot the LFS system is to open the proper console for the new LFS system, which is /.../LINUX_FROM_SCRATCH/dev/console. The only problem caused by a missing /dev/console (aside from the annoying warning message) is that if pre-init encounters an error that occurs before it can open /.../LINUX_FROM_SCRATCH/dev/console (or while opening it), is that you won't see pre-init's error message. The system will just appear to hang and you have to guess at the problem (which is most likely an incorrect path used somewhere). To overcome this problem there are several possibilities. The easiest one is to just create /dev/console (e.g. by using `cp -a' to copy the console device node from your host Linux distro to directory /dev on the target partition). Another solution is to patch the kernel. Responsible for opening /dev/console is the following statement in the file init/main.c of the kernel sources: if (sys_open((const char __user *) "/dev/console", O_RDWR, 0) < 0) printk("Warning: unable to open an initial console.\n"); Just replace "/dev/console" with "/.../LINUX_FROM_SCRATCH/dev/console" and recompile the kernel. Make sure that /.../LINUX_FROM_SCRATCH is the *physical* path on the partition. ########################################################################### 4. Accessing the Full LFS Partition ########################################################################### The method described in this hint is designed to give the illusion that the LFS system has its own dedicated partition. It's intended to look as much as possible like a standard by-the-book LFS system. For that reason, the other files on the partition outside of the /.../LINUX_FROM_SCRATCH directory are not made visible in the LFS system by default. You should treat the partition as if it was not mounted already. If you want to make the other files on it available, mount it somewhere (e.g. /mnt) from inside the LFS system. Although it's no problem to have the same partition mounted several times, there is one small issue that you should be aware of. If you mount the LFS partition from inside the LFS system, you will not only get access to the files outside of /.../LINUX_FROM_SCRATCH, you will also see a 2nd copy of every file from your LFS system. This means, for example, that the command `find / -name bash' will list your LFS system's bash shell twice, one time as "/bin/bash" and a 2nd time as "/mnt/.../LINUX_FROM_SCRATCH/bin/bash" (assuming you mounted the partition on /mnt). This can be annoying and may have undesirable effects, such as archiving files twice when making a full backup. There is a simple yet effective trick to get rid of the redundant directory entries for the LFS system. Simply hide /.../LINUX_FROM_SCRATCH behind a ramfs like this: mount -t ramfs ramfs /mnt/.../LINUX_FROM_SCRATCH This will make all the files in /mnt/.../LINUX_FROM_SCRATCH disappear. If you automount your LFS partition at boot via an entry in /etc/fstab, e.g. because you want to share a directory (e.g. /home) among different systems, you can put a ramfs line corresponding to the above into fstab, too. Just make sure it's listed after the line for the LFS partition. TIP: Keep the /etc/passwd and /etc/group files synchronized among your LFS and other systems that access the same partition. Otherwise a file created by user foo:bar under the new LFS system may appear to be owned by sna:fu when accessing the partition from another system, which can be a security problem. ########################################################################### 5. Elevating the LFS system to / ########################################################################### 5.1 Introduction ---------------- At some point you may decide that you no longer want your LFS system confined to /.../LINUX_FROM_SCRATCH, but want to move it up to the / level in the directory hierarchy. In principle that's an easy task: 1. mkdir /old-root 2. mv /old-root 3. mv / 4. Remove "init=.../pre-init" from kernel command line 5. Enjoy However, there are a few gotchas that you have to watch out for. 5.2 Don't Saw the Branch You're Sitting On ------------------------------------------ The worst mistake you can make is to attempt to perform the above moving from inside the LFS system, by just mounting the partition to /mnt without the hide-behind-ramfs trick, and performing the moves inside /mnt. The problem with this might not be obvious at first glance, but you must not forget that if you mount your LFS partition to /mnt from inside the LFS system, what you see in /mnt/.../LINUX_FROM_SCRATCH is the actual / directory of your LFS system and all of the changes you make there will immediately affect /. Now assume that you do `mv /mnt/.../LINUX_FROM_SCRATCH/lib /mnt/lib', what happens is that instantly your /lib directory disappears. It should be obvious that this is not good. While there are certain tricks that can be used to perform the move from inside the LFS system, the safest way is to boot a different system and use that for the move. It should be obvious that this system must not have any crucial files on the LFS partition, so if your LFS partition is the /usr partition of another system, for example, you can't use that system for doing the move. The best thing to do in that case is to use a boot floppy or boot CD (e.g. Knoppix). 5.3 Don't Break Your Boot Manager --------------------------------- Don't forget that your boot manager sees the physical directory structure of the partition. Dependining on how you set things up, you may not want to move around /boot and/or /.../LINUX_FROM_SCRATCH/boot. 5.4 Don't Touch Special Directories and Files --------------------------------------------- Some directories and files play a special role and must not be moved around. An example of this is the lost+found directory on an ext2fs filesystem. Make sure you don't accidentally move a special file or directory. Remember: Glob patterns are your enemy and * is the worst of the bunch. ########################################################################### Appendix A: Acknowledgements and Changelog ########################################################################### ACKNOWLEDGEMENTS: * Martin Schaffner for his valuable feedback. CHANGELOG: 2004-11-24 * Released version 1.0 2004-11-17 * Hint completely rewritten. * Changed license to CC by-nd 2.0. * Changed versioning scheme to . (starting at 1.0)