Gluster setup notes

Initial configuration

• 2 compute nodes will serve as the initial Gluster servers
• 1 compute node will serve as the initial Gluster client
• Following successfull installation; all nodes will be configured as clients

Requirements

• Packages are provided for 64-bit platforms only
• Packages build cleanly from source for 32-bit Centos 5

Installation steps

• Download source from gluster.com
• Build packages
• wget http://ftp.gluster.com/pub/gluster/glusterfs/2.0/LATEST/glusterfs-2.0.8.tar.gz
• Install dependencies: yum install flex fuse-devel libibverbs-devel
• Build the packages: rpmbuild -ta glusterfs-2.0.8.tar.gz
• Copy the packages out of /usr/src/redhat/RPMS/i386/
• glusterfs-client-2.0.8-1.i386.rpm
• glusterfs-common-2.0.8-1.i386.rpm
• glusterfs-devel-2.0.8-1.i386.rpm
• glusterfs-server-2.0.8-1.i386.rpm
• Servers
• Install fuse on all gluster servers: yum -y install fuse fuse-libs
• Install Gluster rpms on all servers: rpm -Uvh gluster*.rpm
• Create a set of volume configuration templates
• Create a sym link between /etc/glusterfs and /usr/local/etc - otherwise things break
• Create the volume config files
  RAID 0: glusterfs-volgen --name RAID-0 --export-directory /gluster/RAID-0 compute-0-0 compute-0-1
  RAID 1: glusterfs-volgen --name RAID-1 --raid 1 --export-directory /gluster/RAID-1 compute-0-0 compute-0-1
  
• copy /etc/glusterfs/RAID-0-export.vol to /etc/glusterfs/glusterfsd.vol
• copy /etc/glusterfs/RAID-0-mount.vol to /etc/glusterfs/glusterfs.vol
• Start the Gluster servers:
  /etc/init.d/glusterfsd start
• Clients
• Install fuse on all gluster servers: yum -y install fuse fuse-libs
• Install Gluster client and common rpms:
  • glusterfs-client-2.0.8-1.i386.rpm
  • glusterfs-common-2.0.8-1.i386.rpm
• create the client side gluster config directory:
  mkdir -p /etc/gluster
• copy the client config file to /etc/gluster:
  cp RAID-0-mount.vol /etc/gluster/glusterfs.vol
• create the directory where the gluster volumes will be mounted:
  mkdir /gluster
• manually mount the gluster volume and confirm its fully functional:
  mount -t glusterfs /etc/gluster/glusterfs.vol /gluster

Installation issues

• Unless you create a link between /etc/glusterfs and /usr/local/etc/glusterfs glusterfs-volgen will fail
  because it does not know where to put its output files.
• You need to manually copy the glusterfs.vol and glusterfsd files to all of the appropriate servers
• If you create multiple volumes; you have to manually merge the configuration template files
• It's not clear whether gluster expects to use the entire volume specified by glusterfs-volgen
• It's not clear how to control space allocation for gluster volumes

Testing procedures

Performance testing