24 – Kernel Rebuilding and Using Kernel Modules

December 5, 2009 – 10:17 pm

Please note… This information no longer exists at the referenced locations.  This is only a copy of what was available in 2003.

Basic Linux Training™

Kernel Rebuilding and Using Kernel Modules

Henry White

Table of Contents

Prerequisites

Since the pre-compiled kernel that came with your distribution has to be generic enough to support as wide a variety of hardware and other kernel features as possible, it is not tuned to your specific hardware or your specific needs. Obviously, you don’t need drivers for hardware you don’t have, and this does take up a small amount of RAM that is totally useless to you and could be put to better use. Rebuilding the kernel will also give you some insights into how complex the kernel is and the current state of development – if you don’t treat it as a necessary ‘evil’ ;-)

To build the kernel, you will need a number of packages – the specific versions will change with kernel version numbers, so always consult the list ~/linux-2.4.x/Documentation/Changes for the appropriate ones. If these are not consistent with what you have installed, you’ll need to upgrade those before you rebuild the kernel. Kernel source code is running around 20 MB in tarballs and packaged formats; expanded, it will take up well in excess of 110 MB, so don’t attempt this unless you have plenty of free disk space. And it is a good idea to leave the source code on your hard drive after a successful compile so that you can simply apply kernel patches to that source code rather than downloading another 20 MB package ;-) Frankly, once you discover how easy it is to compile from source code, you’ll never be satisfied with a generic kernel again ;-) Further, to a much lesser extent, you can compile other tools, utilities, and applications from source and gain a slight performance boost there as well. Similarly, with the rapid proliferation of Bad Guys on the Internet, and literally tens of millions of unwitting accomplices, having the development tools and the minimal experience required to apply a security patch *might* make you an extraordinarily ‘valuable’ friend or employee at some point in the near future ;-)

Just for reference, the required development packages and versions for the 2.4.x kernels is:

Upgrade to at *least* these software revisions before thinking
you've encountered a bug!  If you're unsure what version you're
currently running, the suggested command should tell you.

Again, keep in mind that this list assumes you are already
functionally running a Linux 2.2 kernel.  Also, not all tools are
necessary on all systems; obviously, if you don't have any PCMCIA
(PC Card) hardware, for example, you probably needn't concern
yourself with pcmcia-cs.

o  Gnu C             2.7.2.3         # gcc --version
o  binutils          2.9.1.0.22      # ld -v
o  util-linux        2.10g           # chsh -v
o  modutils          2.3.10          # insmod -V
o  e2fsprogs         1.18            # /sbin/tune2fs --version
o  pcmcia-cs         3.1.13          # cardmgr -V
o  PPP               2.4.0b1         # pppd --version
o  isdn4k-utils      3.1beta7        # isdnctrl 2<&1|grep version

Do not go by this listing! ALWAYS read and follow the documentation that comes with the kernel version you are working with at the time. Generally, there are no significant changes to these prerequisite development tools within a major version of the kernel – unless, of course, there are bugs discovered ;-) Once you’re familiar with the documentation, it only takes a couple of minutes to check for changes – that’s why files like Changes or Changelog are included and so named.

The faster your microprocessor, and the more physical RAM you have, the quicker the rebuild. On an old ’386 this might take 15-20 hours, dropping progressively as you move up to the latest P4 with might take as little as 20-25 minutes, so plan accordingly.

Kernel Configuration

The best documentation for the kernel comes with the kernel source. In particular:

  • ~/linux-2.4.x/linux/README which includes everything you absolutely, positively must know about downloading and rebuilding your kernel – pay attention to using your home directory for this and NOT using /usr/src!
  • ~/linux-2.4.x/linux/Documentation/Configure.help by Axel Boldt (~760 KB which is why it is NOT included here ;-) and also the fact that the copy with the first release of 2.4.0 includes a slightly dated copy for 2.3.x although there is very little different in 2.4.x)
  • ~/linux-2.4.x/linux/kernel-parameters.txt

My preference is to copy the generic kernel as vmlinuz.generic or better still to append the actual version – for example, vmlinuz-2.2.18 and making those changes in /etc/lilo.conf. You will also need to add a new stanza for the new kernel. For example, the default was

    boot=/dev/hda2
    install=/boot/boot.b
    map=/boot/map
    vga=normal
    lba32
    prompt
    timeout=9000
    image=/vmlinuz
        label=debian
        root=/dev/hda6
        read-only
    ...

which I changed to read:

    ...
    image=/vmlinuz-2.2.18
        label=debian
        root=/dev/hda6
        read-only
    image=/vmlinuz-2.4.0
        label=debian240
        root=/dev/hda6
        read-only
    ...

(In fact, I have had as many as 14 kernel versions in /boot for the 7 distributions I had installed at the time, plus a Win4Lin kernel.) You will have to pay close attention if you expect to share a new kernel on other distributions – simply mount the appropriate partitions and copy the latest lib/modules/2.4.x from the partition where is was installed and compiled to the other partition(s) and check /etc/modules.conf on the other partition(s).

Setting up your kernel configuration (make menuconfig or some GUI version) will take quite a long time on your first attempt, but once that is done you can run the other commands individually or consecutively on one line. What most new users miss are the simple things you assume will be enabled by default (maybe because they are in the generic kernel). One example – ppp is disabled by default, so unless you enable it you won’t be able to connect to the Internet through your ISP ;-) Generally, you’ll want to run this as module anyhow, so you might try adding with the appropriate tools discussed in the Using Modules section before you spend a couple of hour on a rebuild ;-) (If you don’t have ‘modules’ enabled in the kernel, though, this won’t do you any good.) Likewise, check that you have enabled all your hardware and ONLY the hardware you actually have installed! Some of these you can compile directly into the kernel image or as a module – generally it’s best to keep the kernel image as small as possible, so use modules wherever you can.

If you are consistent about changing the names of kernels to include their version number in /boot, and add a new stanza to /etc/lilo.conf, you’ll eliminate 80-90% of the potential problems new users run into on their first compile ;-) (Also, it doesn’t hurt to copy your kernel configuration to your home directory.)

Other than that, the most common problems are simply not having all the packages installed to use gcc. Check the documentation that comes with the kernel source because this will change from time to time as new versions come out.

The finished kernel will be /linux/arch/i386/boot/bzImage which you will copy to /boot. Check /etc/lilo.config and make sure you have a known working kernel included so you can boot if there is a problem with this new kernel, then run /sbin/lilo and reboot.

Trust me! The first time you do this you may be worried sick that you’re going to ruin everything, turn your machine into a worthless pile of junk, and maybe even start World War III ;-) But it’s a LOT easier than you think since most of the work has already been done for you by the kernel hackers. All you need to do is set up the configuration for the hardware you actually have installed and what/how you’re going to be using it, and go through the mechanics outlined above. After the first good rebuild you’ll be strutting around quite proud of yourself – and have good reason to be!

This is a very basic skill that not all Linux users have attempted yet, mainly because the generic kernel is almost always ‘good enough’ ;-) Now you have a kernel that is customized for you and your machine!

Compiling the Kernel

The sequence and description is given in ~/linux-2.4.x/README:

  • download and install the kernel source (I strongly recommend that you download the source code from the FTP site for your distribution of choice. This will already be customized and tested for your distribution, and can save you a LOT of unnecessary grief and frustration. Theoretically, the kernel source is identical, however, the packaging formats are not completely interchangeable, and the distribution may require a few minor tweaks for optimum performance. For example, one of them was recently quoted as having made over 200 changes to the generic kernel source during packaging and testing before it was released to their ‘customers.’ Unless you have the skills and patience to put up with annoying breakage, leave the cutting edge kernel to those who do ;-) NEVER, ever install an untested kernel on production machines!
  • cd to the source directory
  • make mrproper – ONLY if you have rebuilt the kernel before and want to remove all former object files and the former kernel configuration file, otherwise skip this step
  • make menuconfig (or the GUI analogue)
  • make dep
  • make bzImage
  • make modules
  • make modules_install
  • the new kernel image will be ~/linux/arch/i386/boot/bzImage
  • cp that to /boot/vmlinuz-2.4.x
  • edit /etc/lilo.conf as necessary to include the previously known bootable kernel and a new stanza for the new kernel
  • /sbin/lilo
  • reboot
  • verify with uname -a
  • cd back to the kernel source directory
  • make bzdisk to create you boot disk with the new kernel

Note: the last four steps in the actual compile can be run consecutively on one line -

        make dep bzImage modules modules_install

                           or 

       make dep && make bzImage && make modules && make modules_install

It’s that simple! If it doesn’t compile, you probably have one of the wrong versions of a required package (see the Prerequisite section above) or hardware problems – probably RAM. Cut and paste the exact error messages to the list and someone can help you figure it out and get it fixed. (This is also why you need to include the old stanza and the old kernel image that you know boots properly ;-) )

Using Kernel Modules

Many device drives are also available as runtime modules – meaning that they are not compiled directly into the kernel image, but are only called when needed. Only components when are required by the kernel at boot time have to be compiled into the kernel image. Using modules, you can significantly reduce the size of kernel image and it’s footprint in RAM. This will allow older versions of BIOS and boot loaders to boot where they balk if the kernel image is ‘too large’ because everything was compiled into it.

Components and drivers that you do not use often are prime candidates for modules:

  • external filesystems – umsdos, vfat, smbfs, etc.
  • devices that are not required to boot – ppp, sound, printer, etc.

There are a few specialized tools you need to know how to use with modules:

  • insmod – deprecated utility to load or ‘install’ modules; the preferred and recommended tool is modprobe
  • rmmod – utility to ‘remove’ modules
  • depmod – creates /lib/modules/<version>/modules.dep which controls which modules are loaded and their dependencies
  • modprobe – load or installs modules, checks dependencies, preferred over insmod (note: if you want to permanently remove the module on all future boots, edit /etc/modules which are loaded by kerneld when needed)
  • lsmod – ‘list’ currently installed modules, their size and how they are used

You will have to read the appropriate manpages to understand the options and switches and how to use these utilities.

Installing the New Kernel

As with all things Unix, there is more than one way to install the newly compiled kernel. The method outlined above is the preferred and traditional way – particularly for new users who need to go through the sequence step-by-step so that they completely understand the process. If the new kernel fails to boot, it is easy to retrace your steps and find the problem.

The other way, which I consider to be much too risky for inexperienced user, is to substitute the Makefile target make bzlilo for the copying and editing at the end. This will rename the old kernel image to /boot/vmlinuz.old and copy the new bzImage to /boot/vmlinuz and run /sbin/lilo. (You will have to complete the process yourself and make modules ; modules_install as usual.) The problem with this is that it presumes a fixed naming convention that is not compatible with this naming convention.

Assignments

Online:

Copyright © 1997-2003 Henry White. All Rights Reserved.
Reproduction or redistribution without prior written consent is strictly prohibited. Address comments and inquiries to info@basiclinux.net

Sorry, comments for this entry are closed at this time.

Proudly powered by WordPress. Entries (RSS) and Comments (RSS). Silver Light 3-Column theme by Bob, modified by Ms. Crafty.