>Any tips on this stuff would be greatly appreciated.  Are any of you 
>running a 64-bit Linux OS on a multi-processor Opteron machine?

I run a dual Opteron system using LinuxBIOS with AMD64 Fedora Core 2 and
now Fedora Core 3.  I haven't installed Fedora Core 4 yet, but I'm
positive that AMD64 Debian would be better and plan to install it next.

A cluster of single core, dual (two socket) Opteron systems may be a lot
cheaper than dual core Opterons on four or eight socket mainboards.

Either way, Tyan is the best choice for multiple socket mainboards.
They also have the best support for LinuxBIOS and will supply LinuxBIOS
on all mainboards they sell.  (Disclaimer: I have no connection with
Tyan, other than using a few of their mainboards.)

The Cell processor with its central PPC core and eight Synergistic
Processing Units (SPU) is an interesting alternative to true multi-core
CPUs.  Linux runs on the central PPC core and the central core controls
what the eight SPUs do.

Sincerely,

Ken Fuchs <kfuchs at winternet.com>