Upgrading to FreeBSD 5.3

With a few days leave available, it seemed a good time to upgrade the FreeBSD server, crimson, from 4.11 to 5.3. I'd held off migrating to the newer version due to my experiences with the earliest release of 5.0. By coincidence, I also had just acquired a new laptop, an IBM Thinkpad T42, so I also installed 5.3 on this device.

FreeBSD 5.3 on a Compaq Deskpro 5120

The web/mail server is hosted on a very old (and slow) hardware platform. Its memory is now 64MB, with a hard drive of 8GB. I could just upgrade to 5.3 using cvsup, but I decided to buy a new, bigger (80GB), disk for the 5.3 install. This was partly so I could use the new UFS2 filesystem that comes with 5.3 and partly because it offered less risk if something went horribly wrong in the 5.3 install.

Building the server

I downloaded a snapshot i386 iso from the FreeBSD snapshot location and burnt it to a CD. Since 5.4 was so close to arriving I took the chance of starting from 5.3-STABLE. To perform the actual install, I temporarily put the new hard disk in silver (normally a Windows 2000 machine). Its 2GHz processor should also help the install run a little faster.

The 5.3-STABLE version came up fine. I named it crimson5 as it would have to coexist with the 4.11 version of crimson for a while. I created the following partition scheme (in one slice):

  Filesystem  1K-blocks    Used    Avail Capacity  Mounted on
  /dev/ad0s1a    126702   52384    64182    45%    /
  devfs               1       1        0   100%    /dev
  /dev/ad0s1g  30462636  219134 27806492     1%    /home
  /dev/ad0s1h  40401964       4 37169804     0%    /opt
  /dev/ad0s1d    253678      12   233372     0%    /tmp
  /dev/ad0s1f   4058062 1444964  2288454    39%    /usr
  /dev/ad0s1e    247726    4392   223516     2%    /var

Installing ports

The snapshot mini-ISO contains one package - perl-5.8.6, so once the main install is complete, everything else has to be installed over the Internet. I used /stand/sysinstall to perform the initial package install, but I had trouble providing a release name (specified via the Options menu under Custom) that corresponded to a package set on the FreeBSD servers; I figured I could use 5-STABLE, but that didn't work. The only value that persuaded /stand/sysinstall/ to continue was 5.3-RELEASE.

The original crimson had X, which I didn't really need, so the only packages I downloaded were:

  apache
  cvsup-without-gui
  emacs
  gmake
  perl
  portaudit
  portupgrade
  procmail
  py24-bsddb
  python
  qpopper
  rsync
  ruby
  ruby18-bdb1
  sendmail

Needless to say, these packages pulled in a lot of others as dependencies. Emacs ended up downloading elements of X, which kind of negated my simplification. However, I re-built emacs using WITHOUT_X11 so I could safely delete:

  Xaw3d
  jpeg
  tiff
  libungif
  png
  imake
  xorg-libraries
  freetype2
  fontconfig
  pkgconfig (?)

I didn't really want perl, but it seems to be needed by apache (I have my doubts).

Now that cvsup and portupgrade were available, I could bring the ports tree up-to-date and upgrade any out of date applications. I also created a custom kernel for crimson, which was basically GENERIC without SCSI, most network adapters and USB. However, I didn't install the new kernel; I wanted to see if the GENERIC kernel would boot up in the target hardware first.

Testing

Last thing before swapping the hard drive to the target hardware was to rename crimson5 as crimson. This involves changing /etc/rc.conf, /etc/hosts and /etc/dhclient.conf (which sends the host name to the DHCP server).

To ensure that my many visitors (ha!) had an uninterrupted experience, before shutting down crimson, I booted the backup server (chrome), and set the NAPT of the Alcatel Speedtouch 510 router to direct HTTP and SMTP traffic to it. Crimson was shutdown, the hard drive replaced and rebooted. Everything worked just fine, once I had told the Compaq firmware to save the change of disk size. Even the custom kernel worked, once I installed it and rebooted.

I did encounter a slight problem with sendmail. I had incorrectly created the /etc/mail/local-host-names file (which defines the domains to accept mail for) with the wrong name (local_host_names). When I tested the mail server with an incoming mail message from outside the domain (via my ISP mail account), the message bounced madly between crimson and the ISP, until:

  554 5.4.6 Too many hops 26 (25 max): from <me@isp.com> via
  localhost, to <testing@hydrus.org.uk>

Once the file name was fixed, the problem vanished. Then testing the web server revealed another, more serious, problem. Crimson froze completely when attempting to deliver the home page; the hydrus logo half displayed and then all activity stopped. The console was unresponsive, as were any open ssh sessions. Strangely, the ssh sessions did not die (at least not until I power cycled the machine). My immediate suspicion fell upon the lance (PCNet/PCI) network adapter, as I had previously seen one message like:

  lnc0: Start of packet found before end of previous in transmit ring -- Resetting  
  lnc0: Missed packet -- no receive buffer

However, that message had been issued a long time before the apache testing. Also, I'd seen Missed packet -- no receive buffer messages in 4.x, which had not seemed to cause problems. Having been lead up the garden path many times based on hunches, I decided to test all the elements I could.

Firstly, could I reproduce the problem? Yep, running a big machine-to-machine copy using rsync and hitting the apache server would make crimson freeze in a very short time. Rsync on it's own seemed to be fine. OK, was the problem the custom kernel? I rebooted back into GENERIC and tested again; another freeze. Was this some 5.3 related problem, independent of crimson? I put the hard disk back into silver; couldn't make it freeze, with or without loading ACPI (crimson didn't support ACPI).

OK, now to try a new network adapter. I installed the spare Netgear FA331 adapter I had (which was nice). In FreeBSD, this uses the sis driver. I re-installed the hard drive in crimson, re-booted and tested. No freeze. Hmm, for once the initial hunch was correct. Since this test used the GENERIC kernel, I rebuilt and installed a custom CRIMSON kernel, specifying the sis driver, rather than the lnc driver. Testing with the custom kernel also seemed fine. I decided to let the system bed down for a week or so before doing anything to the backup server. Finally, the new server was up and running.

[N.B. 3rd March, 2008] I can no longer reproduce the problem with the PCI/PCNet card, once I'd replaced the lnc driver by if_le. The lnc driver is deprecated. However, many other things have changed; now on FreeBSD 6.3 and the network card is connected to a different switch - still my money's on the driver.

Notes

It's apparent that the new version of gcc (3.4.2) is a lot slower at compilation (especially when using the -MM -MG options); a kernel build on the crimson platform used to take about one hour for 4.11; for 5.3 it takes about three hours. I therefore set the variable NO_MODULES=true in the /etc/make.conf file, since crimson uses no modules at all. This brings the buildkernel time down to just less than one hour.

FreeBSD 5.3 on a IBM Thinkpad T42

The install was pretty easy, once I'd found a site which saved all the hard work.

Disk Partitioning

In preparation, I shrank the Windows XP partition to 25GB, leaving about 8GB for FreeBSD. I used Knoppix to shrink the NTFS partition, which works, but then some arithmetic is required (and some prayer) since the original partition must be deleted and then recreated with a new, smaller size using Linux fdisk. The IBM recovery partition takes a massive 4GB - I've since requested the Windows recovery CDs from IBM so that I can use this space. The partition/slice layout as shown by FreeBSD fdisk is below:

  ******* Working on device /dev/ad0 *******
parameters extracted from in-core disklabel are:
cylinders=77520 heads=16 sectors/track=63 (1008 blks/cyl)

Figures below won't work with BIOS for partitions not in cyl 1
parameters to be used for BIOS calculations are:
cylinders=77520 heads=16 sectors/track=63 (1008 blks/cyl)

Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 7 (0x07),(OS/2 HPFS, NTFS, QNX-2 (16 bit) or Advanced UNIX)
    start 63, size 52919937 (25839 Meg), flag 80 (active)
        beg: cyl 0/ head 1/ sector 1;
        end: cyl 1023/ head 29/ sector 63
The data for partition 2 is:
sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
    start 52920000, size 15597225 (7615 Meg), flag 0
        beg: cyl 1023/ head 255/ sector 63;
        end: cyl 1023/ head 254/ sector 63
The data for partition 3 is:
sysid 18 (0x12),(Compaq diagnostics)
    start 68523840, size 9616320 (4695 Meg), flag 0
        beg: cyl 1023/ head 255/ sector 63;
        end: cyl 1023/ head 254/ sector 63
The data for partition 4 is:
<UNUSED>

According the miraclenet web site, if the IBM recovery partition is not on the second partition, the recovery won't work. Naturally, I didn't read that bit until it was too late. I sure hope the recovery CDs arrive before I have any problems with the Windows installation.

The FreeBSD 5.3 snapshot ISO booted and installed without any problems. Well, there was one small one: the boot hung for a little while during the probe for disks, and produced the error message: "ata1-slave: FAILURE - ATAPI_IDENTIFY timed out". It doesn't seem to cause any other problems, which is just as well, as there is no obvious fix according to Google. Anyway, here's the partitions defined within the FreeBSD slice:

  Filesystem  1K-blocks    Used   Avail Capacity  Mounted on
  /dev/ad0s2a    126702   36290   80276    31%    /
  devfs               1       1       0   100%    /dev
  /dev/ad0s2g   1847458  285462 1414200    17%    /home
  /dev/ad0s2d    247726      16  227892     0%    /tmp
  /dev/ad0s2f   4058062 1759546 1973872    47%    /usr
  /dev/ad0s2e    247726   14514  213394     6%    /var

I set the swap partition up as 1GB, probably overkill.

I had elected not to install the FreeBSD boot manager, since I intended to use the Windows boot manager instead. When I tried to reboot the laptop back into Windows, my heart nearly stopped. The message "No operating system found" appeared. Had I accidently trashed the NTFS partition? I booted from the Knoppix CD to have a look at the partition table; the new FreeBSD partition had been left marked as bootable. I set the NTFS partition as bootable, and restarted. This time Windows came up.

To use the Windows boot manager, copy the file /boot/boot1 from the FreeBSD install CD to the C:\ directory under Windows. I called the file FreeBSD.bin. [Note: /boot/boot1 can only be used when Windows and FreeBSD reside on the same disk]. Then, add C:\FreeBSD.bin to the end of c:\boot.ini, like so:

  [boot loader]
  timeout=30
  default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
  [operating systems]
  multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP  Professional"   /fastdetect
  c:\FreeBSD.bin="FreeBSD"

Since the wireless card needed some additional software (see below), I used the inbuilt ethernet port to download the initial packages. The in-built port is an Intel(R) PRO/1000 Network Connection device, which uses the em driver,

Wireless Card Setup

The Thinkpad T42 is Centrino-based and therefore has an in-built wireless adapter, the Intel(R) PRO/Wireless 2200BG MiniPCI. Support for this network adapter is not available in the standard 5.3 release, but the driver, firmware and instructions for setup can be found on Damien Bergamini's site (no longer).

Building the driver is straightforward:

  # tar xzvf iwi-freebsd-VERSION.tgz
  # cd iwi-freebsd-VERSION/
  # make && make install

If everything goes well, it will have created the following files:

  /boot/kernel/if_iwi.ko
  /usr/sbin/iwicontrol
  /usr/share/man/man4/iwi.4.gz
  /usr/share/man/man8/iwicontrol.8.gz

The card requires firmware, available separately. Damien supplies it as a FreeBSD package:

  # pkg_add http://damien.bergamini.free.fr/ipwfw/FreeBSD/ipw-firmware-1.1.tbz

While the instructions ensured I had a working wireless card, it wasn't obvious how the settings could be made to happen at boot time. It turns out to be fairly easy. I added the following lines to /boot/loader.conf:

  if_iwi_load="YES"
  snd_ich_load="YES"

This ensure that the module for 2200BG wireless adapter (iwi) is loaded at boot time. The other line loads the driver for the on-board sound card.

To configure the iwi0 adapter, I created the file /etc/start_if.iwi0 with the following lines:

  iwicontrol -i iwi0 -d /usr/local/libdata/if_iwi -m bss
  ifconfig iwi0 ssid default wepkey 0xdeadbeef11 wepmode on

The first command ensures the firmware is loaded into the wireless adapter, and the second sets the wep parameters. I see the error message "iwi0: unknown authentication state 1" at boot time but, once again, there seems to be no ill effects.

X Stuff

The on-board graphics adapter is an ATI Radeon Mobility M7 LW. I specified radeon as the driver in the xorg.conf file (which I had just copied from my other Thinkpad FreeBSD machine). Once X was running, I noticed the mouse pointer was sluggish when moving the mouse using a Microsoft USB mouse (an IntelliMouse Explorer 4.0 USB/PS2 compatible). The touchpad moved the mouse ok. I swapped the mouse for the version I used on the Thinkpad T30 (an IntelliMouse Optical 1.1A), which worked perfectly. Ah well, I didn't really like that newer mouse anyway.

Loading modules

While testing which modules I needed for the sound card, I noticed that loading modules after FreeBSD was booted could cause either the wireless port or the USB mouse to stop working. However, loading at boot time via /boot/loader.conf didn't cause these problems.

Upgrading the Thinkpad T42 to FreeBSD 6.0

Subsequently, I installed FreeBSD 6.0 (at the RC-1 stage) on the IBM laptop. This was mostly because 6.0 includes the iwi drivers, which meant that I could forget about that extra step of building/installing the iwi drivers when performing any further kernel changes.

The upgrade was mostly painless. The iwi driver worked ok, once I had included the wlan_wep module in the build process, as this is now separate. I no longer needed the start_if.iwi0 script, as its work could be managed via rc.conf, with the additional lines:

iwi_enable="YES"
ifconfig_iwi0="DHCP ssid default nwkey 0xdeadbeef99"

The other change affected /etc/make/conf, where a number of the NOXXX make options have been renamed to NO_XXX. Note that the mergemaster setup takes quite a long time, as there are many changes to the /etc/ files. I was glad to see that the "ata1-slave: FAILURE - ATAPI_IDENTIFY timed out" error at boottime had vanished.

Here's the new /etc/make.conf for 6.0:

# make.conf for OPAL
#
# Modification History:
# Date      Who
# 20050311  mpw
#   Added options to prevent unused functionality being built
#   Added BATCH=true
# 20051014 mpw
#   6.0 changes: added iwi wlan_wep to modules
#                NOXXX -> NO_XXX

# standard CFLAGS
CFLAGS= -O -pipe

# no profiled libraries
NO_PROFILE=true

# The following modules, only, are required
MODULES_OVERRIDE= sound acpi linux linprocfs ntfs iwi wlan_wep

# omit these elements from buildkernel/buildworld
NO_BIND=true
NO_IPFILTER=true
NO_PF=true
NO_AUTHPF=true
NO_ATM=true
NO_LPR=true
NO_VINUM=true
NO_BLUETOOTH=true
NO_I4B=true
NO_OBJC=true
# can't use AAAA queries against the Alcatel DNS
NO_INET6=true

# Build ports in unattended mode
BATCH=true

# added by use.perl 2005-03-09 12:06:02
PERL_VER=5.8.6
PERL_VERSION=5.8.6