Virtual HardwareI started off with a small VM to build the image on. I can always re-size the virtual hardware later on if needed. The base VM I started with was 1 CPU, 2 GB Memory and 8 GB of disk. If you are installing to physical, you probably need to follow the recommended sizing guide for Pivotal HD at http://docs.gopivotal.com/pivotalhd/InstallingPHDUsingtheCLI.html.
Installing CentOSNext, I installed CentOS 6.4 using the minimal ISO which can be downloaded from http://vault.centos.org/6.4/isos/x86_64/CentOS-6.4-x86_64-minimal.iso. Either burn it to a CD, or USB stick (http://wiki.centos.org/HowTos/InstallFromUSBkey), or use your virtualization software to attach the ISO to the VM and boot from it.
|Boot menu after booting from CentOS 6.4 Minimal ISO|
|Configuring the Host Name|
|Make sure to connect automatically to the network|
Next you need to select your timezone, and enter a root password (don't forget it!).
After selecting your root password and clicking next, you'll be presented with a screen to customize your disk partitioning. Choose "Use all space", check the "Review and modify partitioning layout" check-box, and click "Next". In the resulting dialog, make sure that the "lv_swap" isn't more than 2048, and re-size it if it is so that you can use the rest of the space for lv_root. When you are satisfied with the the partitioning, click "Next" to continue.
Click "Next" in the next dialog to just install the GRUB boot loader with it's default settings. Now you can grab a quick coffee and wait for the install to finish. Remove the install media from the machine, and click the "Reboot" button.
Further CustomizationWhen the machine reboots and you log back in as root, whatever you do, do _not_ call yum update! This will upgrade your image to a later version of CentOS which will not work with Pivotal HD. We do want to install a few more packages, though, using the following command:
yum -y install perl wget ntp ed samba-client cifs-utils
The samba-client and cifs-utils packages aren't required, but they are nice to have if you are in a Windows environment and need to mount shares.
Windows SharesIf you are using Windows shares, and you want to be able to resolve Windows machine names, you'll need to edit /etc/nsswitch.conf to add WINS name resolution. You can call the following to change your nsswitch.conf file:
sed -i "s/^hosts:.*/hosts: files wins dns/" /etc/nsswitch.conf
SELinuxPivotal HD requires that SELinux be set to permissive or disabled. To disable SELinux, issue the following command:
sed -i 's/SELINUX=[a-z]*/SELINUX=disabled/' /etc/selinux/config
Disable iptables and ip6tablesPivotal HD currently requires that the local firewall be disabled as well. You can disable the firewall with the following commands:
chkconfig iptables off chkconfig ip6tables off service iptables stop service ip6tables stop
Enable NTPTime synchronization is very important for a stable cluster. To ensure consistent times across the cluster, we'll use NTP. You can enable NTP with the following commands:
chkconfig ntpd on service ntpd start
Multicast DNS with avahiSetting up a DNS server just for a Pivotal HD cluster can be a pain. Also, even if you do have a DNS server, you may not have the ability to add new entries to it. So, if you'd rather avoid having to setup or modify DNS for your cluster, you can use multicast DNS with a service called avahi, and a corresponding name resolution model called mdns.
Multicast DNS allows you to resolve hosts using a ".local" domain. So if you set your machine's host name to foo, then you could ping "foo.local" from another machine that has mDNS name resolution enabled.
For you Mac users out there, multicast DNS is how Mac Bonjour works.
The avahi service relies on another service called D-Bus, which isn't in the default CentOS repos. To get D-Bus, you'll have to add another yum repo to your machine. The blog post at http://theengguy.blogspot.com/2013/02/mdns-centos-63.html provides instructions that are close to what you need to do to add the ATRPMs repo, but below are the steps I had to follow to get it to work:
rpm --import http://packages.atrpms.net/RPM-GPG-KEY.atrpms rpm -ivh http://dl.atrpms.net/el6.4-x86_64/atrpms/stable/atrpms-repo-6-7.el6.x86_64.rpm sed -i 's/enabled=1/enabled=0/' /etc/yum.repos.d/atrpms.repo
Next, we install the avahi and nss-mdns packages:
yum -y --enablerepo=atrpms install nss-mdns avahi
Finally, we enable the avahi service, and start D-Bus and avahi up.
chkconfig avahi-daemon on service messagebus start service avahi-daemon start
The above instructions enabled this machine to respond to mDNS requests, but we also want to be able to resolve names on this machine via mDNS. To do that, we need to modify the /etc/nsswitch.conf file. Remove the "wins" part of the command below if you don't need name resolution using WINS.
sed -i "s/^hosts:.*/hosts: files mds4_minimal [NOTFOUND=return] wins dns mdns4/" /etc/nsswitch.conf
Enable password-less SSH between cluster hostsTo allow Pivotal HD to install, you will need to enable password-less SSH between the cluster hosts. First you need to generate keys with the following command:
Next, if you are building a virtual machine base template, you can pre-share the key you just generated with the following commands:
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys chmod 600 ~/.ssh/authorized_keys
Now, when you clone the virtual machine, you will have the same keys for root and the same public key in authorized_keys for all your machines. If you are installing multiple machines by hand, then you can use the ssh-copy-id script to copy your key to each of the machines in your cluster by hand (replace or set $A_MACHINE with the IP or host name of the host you want to copy your public key to):
Install Oracle 1.7 JDK RPMPivotal HD requires Oracle JDK 1.7 to be installed. You will need to download it using a browser, as Oracle requires you to accept their licensing agreement before downloading. You can download the 1.7 JDK from links on http://www.oracle.com/technetwork/java/javase/downloads/index.html. Download the 64 bit RPM to make installation easier. After downloading, you can scp (or perhaps pscp for Windows) the file to your new machine.
After getting the file up to your machine, you can install the JDK with the following command in the directory you uploaded the RPM to:
yum -y localinstall jdk-7u*-linux-x64.rpm
Update alternativesUnfortunately, Oracle's RPM doesn't properly update the alternatives links on CentOS. To update those links, issue the following command:
alternatives --install /usr/bin/java java /usr/java/default/bin/java 1 \ --slave /usr/bin/jar jar /usr/java/default/bin/jar \ --slave /usr/bin/javac javac /usr/java/default/bin/javac \ --slave /usr/bin/javadoc javadoc /usr/java/default/bin/javadoc \ --slave /usr/bin/javaws javaws /usr/java/default/bin/javaws \ --slave /usr/bin/jcontrol jcontrol /usr/java/default/bin/jcontrol
Additional Steps for Virtual Machine TemplateIf you are creating a base template to use for Virtual based installs, then you will need to perform the following steps.
Install VMware ToolsIn vSphere (or Fusion or Workstation), execute the function to "Install VMware Tools". For Fusion, refer to the manual instructions at http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003417, for Workstation refer to http://pubs.vmware.com/workstation-10/index.jsp?topic=%2Fcom.vmware.ws.using.doc%2FGUID-012378D8-A995-4B78-AAD3-5A4223C4093E.html, and for ESX refer to http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1014294.
Which ever method you use, you will need to execute the following commands to install VMware Tools in your CentOS machine:
mount /dev/cdrom /mnt cd /tmp tar zxvf /mnt/VMwareTools-*.tar.gz cd /vmware-tools-distrib ./vmware-install.pl umount /mnt rm -rf vmware-tools-distrib
Make Networking work after CloningIf you are building a VM for cloning, you will want to execute the following commands to make networking work when the VM is started after a clone:
sed -i "/UUID=.*/d" /etc/sysconfig/network-scripts/ifcfg-eth0 sed -i "/HWADDR=.*/d" /etc/sysconfig/network-scripts/ifcfg-eth0 rm /etc/udev/rules.d/70-persistent-net.rules
If you start your base image again for any reason after this step, you will need to execute the last line of the commands above as the referenced file is generated after every boot.
CloneAt this point you can clone the VM you built above as many times as you need. After starting up the cloned VM, you will want to edit the host name using the following commands:
If you need to change the host name for the machine you are working on, you can do so with the following commands (either replace $NEWFQDN with the new fully qualified host name, or set this variable before executing the commands):
sed -i 's/HOSTNAME=.*/HOSTNAME=$NEWFQDN/' /etc/sysconfig/network hostname $NEWFQDN
If you are using avahi, you need to restart it to get the new host name picked up:
service avahi-daemon restart