> Ted Maul wrote: > > Does anyone have experience with gfs, zfs, or lustre? I'd like to > add/remove nodes at will and I'm wondering which fs would work best. I'm > not looking for blistering performance, just something that's fairly > reliable and easily scaled (i.e., plug in another server and have it's > available storage added in minutes). Suggestions outside of those three > are welcome. > On Thu, Apr 02, 2009 at 11:35:31AM -0500, Elvedin Trnjanin wrote: > I'm taking a look at GlusterFS (not the same as lustre) which does seem to > have great performance and is reliable, although I haven't tested that due > to lack of hardware. GlusterFS is not currently capable of online expansion. Because most of its intelligence is on the client side, the clients' glusterfs.vols need to be updated, and then the filesystem remounted (so all workload using the filesystem must be stopped). It is, however, a nicely-designed distributed filesystem, in my opinion. The folks at Z RESEARCH are also extremely responsive. Ted: Regarding the other filesystems mentioned, GFS cannot be grown online (as far as I know); ZFS is not cluster-aware and is limited to a single server; and Lustre _may be capable of online expansion, but the process is not pretty (see http://manual.lustre.org/manual/LustreManual16_HTML/LustreOperatingTips.html#50548871_22527). Lustre is also really best used for very large reads and writes, as small I/O can overwelm its metadata servers. If you want elegant online expansion, check out Isilon. It comes at a price premium, however. HTH, Gabe -- Gabe Turner gabe at msi.umn.edu HPC Systems Administrator, University of Minnesota Supercomputing Institute http://www.msi.umn.edu