Creating a base CentOS 6.4 Image for Pivotal HD Installation
Pivotal HD 2.0 is simple to install once you have a proper base to install to. However, there are some prerequisites and some things you can do to make the install process simpler. Although these instructions were mainly for a virtual machine based install, they can be used for a physical machine based install by just ignoring certain steps.
After selecting the first option in the boot menu, you'll be asked if you want to test the install media. Skip it unless you want to wait an long time, and you are unsure the install ISO is valid.
The install will switch into graphics mode. Just select whatever makes sense for your install from the language and keyboard dialogs.
Next, select "Basic Storage Devices" when asked "What types of devices will your installation have?"
You will probably see a dialog pop up that is warning you that your disk may contain data and you are likely to lose it. If this is a new VM with a new disk, then you are fine to click "Yes, discard my data". If this is a physical server, double check that you have the correct disk shown in the dialog, and click "Yes" only if you are sure it is ok to wipe out. If not, select "No, keep any data" and move on. You usually only would use "No" if you have another operating system on the same disk that you want to retain.
Next, give your machine a host name. I set mine to centos6base.local, but you are free to use whatever you like here. If you are installing this machine as a virtual machine that you will clone, use some host name you are likely not to use for your cluster. The most important thing to do here is to click the "Configure Network" button.
In the resulting dialog, select the "System eth0" adapter, and click the "Edit" button. Check the "Connect Automatically" check-box, and click "Apply" then "Close" and then "Next" to continue.
You could certainly configure a static IP address here if you know you are already going to need it, but leaving the adapter to configure itself via DHCP is a reasonable starting point if you are on a trusted network. If this is a Virtual Machine image you are building, then it may also make sense to start with DHCP for the time being, and then change the machine later on to use a static IP if needed. If you set a static address, make sure to set the proper DNS server entries to allow name resolution to work.
Next you need to select your timezone, and enter a root password (don't forget it!).
After selecting your root password and clicking next, you'll be presented with a screen to customize your disk partitioning. Choose "Use all space", check the "Review and modify partitioning layout" check-box, and click "Next". In the resulting dialog, make sure that the "lv_swap" isn't more than 2048, and re-size it if it is so that you can use the rest of the space for lv_root. When you are satisfied with the the partitioning, click "Next" to continue.
You will likely be presented with warning that the disk is going to be formatted and all data removed. This is your last chance to save anything you might have on that disk that you want to keep. If you are good with wiping out that disk, click "Format", and then "Write changes to disk".
Click "Next" in the next dialog to just install the GRUB boot loader with it's default settings. Now you can grab a quick coffee and wait for the install to finish. Remove the install media from the machine, and click the "Reboot" button.
The samba-client and cifs-utils packages aren't required, but they are nice to have if you are in a Windows environment and need to mount shares.
Multicast DNS allows you to resolve hosts using a ".local" domain. So if you set your machine's host name to foo, then you could ping "foo.local" from another machine that has mDNS name resolution enabled.
For you Mac users out there, multicast DNS is how Mac Bonjour works.
The avahi service relies on another service called D-Bus, which isn't in the default CentOS repos. To get D-Bus, you'll have to add another yum repo to your machine. The blog post at http://theengguy.blogspot.com/2013/02/mdns-centos-63.html provides instructions that are close to what you need to do to add the ATRPMs repo, but below are the steps I had to follow to get it to work:
Next, we install the avahi and nss-mdns packages:
The above instructions enabled this machine to respond to mDNS requests, but we also want to be able to resolve names on this machine via mDNS. To do that, we need to modify the /etc/nsswitch.conf file. Remove the "wins" part of the command below if you don't need name resolution using WINS.
Next, if you are building a virtual machine base template, you can pre-share the key you just generated with the following commands:
Now, when you clone the virtual machine, you will have the same keys for root and the same public key in authorized_keys for all your machines. If you are installing multiple machines by hand, then you can use the ssh-copy-id script to copy your key to each of the machines in your cluster by hand (replace or set $A_MACHINE with the IP or host name of the host you want to copy your public key to):
After getting the file up to your machine, you can install the JDK with the following command in the directory you uploaded the RPM to:
Which ever method you use, you will need to execute the following commands to install VMware Tools in your CentOS machine:
If you start your base image again for any reason after this step, you will need to execute the last line of the commands above as the referenced file is generated after every boot.
If you need to change the host name for the machine you are working on, you can do so with the following commands (either replace $NEWFQDN with the new fully qualified host name, or set this variable before executing the commands):
If you are using avahi, you need to restart it to get the new host name picked up:
Virtual Hardware
I started off with a small VM to build the image on. I can always re-size the virtual hardware later on if needed. The base VM I started with was 1 CPU, 2 GB Memory and 8 GB of disk. If you are installing to physical, you probably need to follow the recommended sizing guide for Pivotal HD at http://docs.gopivotal.com/pivotalhd/InstallingPHDUsingtheCLI.html.Installing CentOS
Next, I installed CentOS 6.4 using the minimal ISO which can be downloaded from http://vault.centos.org/6.4/isos/x86_64/CentOS-6.4-x86_64-minimal.iso. Either burn it to a CD, or USB stick (http://wiki.centos.org/HowTos/InstallFromUSBkey), or use your virtualization software to attach the ISO to the VM and boot from it. |
| Boot menu after booting from CentOS 6.4 Minimal ISO |
 |
| Configuring the Host Name |
 |
| Make sure to connect automatically to the network |
Next you need to select your timezone, and enter a root password (don't forget it!).
After selecting your root password and clicking next, you'll be presented with a screen to customize your disk partitioning. Choose "Use all space", check the "Review and modify partitioning layout" check-box, and click "Next". In the resulting dialog, make sure that the "lv_swap" isn't more than 2048, and re-size it if it is so that you can use the rest of the space for lv_root. When you are satisfied with the the partitioning, click "Next" to continue.
 |
| Reviewing partitioning |
Click "Next" in the next dialog to just install the GRUB boot loader with it's default settings. Now you can grab a quick coffee and wait for the install to finish. Remove the install media from the machine, and click the "Reboot" button.
Further Customization
When the machine reboots and you log back in as root, whatever you do, do _not_ call yum update! This will upgrade your image to a later version of CentOS which will not work with Pivotal HD. We do want to install a few more packages, though, using the following command:yum -y install perl wget ntp ed samba-client cifs-utils
The samba-client and cifs-utils packages aren't required, but they are nice to have if you are in a Windows environment and need to mount shares.
Network
Windows Shares
If you are using Windows shares, and you want to be able to resolve Windows machine names, you'll need to edit /etc/nsswitch.conf to add WINS name resolution. You can call the following to change your nsswitch.conf file:sed -i "s/^hosts:.*/hosts: files wins dns/" /etc/nsswitch.conf
SELinux
Pivotal HD requires that SELinux be set to permissive or disabled. To disable SELinux, issue the following command:sed -i 's/SELINUX=[a-z]*/SELINUX=disabled/' /etc/selinux/config
Disable iptables and ip6tables
Pivotal HD currently requires that the local firewall be disabled as well. You can disable the firewall with the following commands:chkconfig iptables off chkconfig ip6tables off service iptables stop service ip6tables stop
Enable NTP
Time synchronization is very important for a stable cluster. To ensure consistent times across the cluster, we'll use NTP. You can enable NTP with the following commands:chkconfig ntpd on service ntpd start
Multicast DNS with avahi
Setting up a DNS server just for a Pivotal HD cluster can be a pain. Also, even if you do have a DNS server, you may not have the ability to add new entries to it. So, if you'd rather avoid having to setup or modify DNS for your cluster, you can use multicast DNS with a service called avahi, and a corresponding name resolution model called mdns.Multicast DNS allows you to resolve hosts using a ".local" domain. So if you set your machine's host name to foo, then you could ping "foo.local" from another machine that has mDNS name resolution enabled.
For you Mac users out there, multicast DNS is how Mac Bonjour works.
The avahi service relies on another service called D-Bus, which isn't in the default CentOS repos. To get D-Bus, you'll have to add another yum repo to your machine. The blog post at http://theengguy.blogspot.com/2013/02/mdns-centos-63.html provides instructions that are close to what you need to do to add the ATRPMs repo, but below are the steps I had to follow to get it to work:
rpm --import http://packages.atrpms.net/RPM-GPG-KEY.atrpms rpm -ivh http://dl.atrpms.net/el6.4-x86_64/atrpms/stable/atrpms-repo-6-7.el6.x86_64.rpm sed -i 's/enabled=1/enabled=0/' /etc/yum.repos.d/atrpms.repo
Next, we install the avahi and nss-mdns packages:
yum -y --enablerepo=atrpms install nss-mdns avahi
Finally, we enable the avahi service, and start D-Bus and avahi up.
chkconfig avahi-daemon on service messagebus start service avahi-daemon start
The above instructions enabled this machine to respond to mDNS requests, but we also want to be able to resolve names on this machine via mDNS. To do that, we need to modify the /etc/nsswitch.conf file. Remove the "wins" part of the command below if you don't need name resolution using WINS.
sed -i "s/^hosts:.*/hosts: files mds4_minimal [NOTFOUND=return] wins dns mdns4/" /etc/nsswitch.conf
Enable password-less SSH between cluster hosts
To allow Pivotal HD to install, you will need to enable password-less SSH between the cluster hosts. First you need to generate keys with the following command:ssh-keygen
Next, if you are building a virtual machine base template, you can pre-share the key you just generated with the following commands:
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys chmod 600 ~/.ssh/authorized_keys
Now, when you clone the virtual machine, you will have the same keys for root and the same public key in authorized_keys for all your machines. If you are installing multiple machines by hand, then you can use the ssh-copy-id script to copy your key to each of the machines in your cluster by hand (replace or set $A_MACHINE with the IP or host name of the host you want to copy your public key to):
ssh-copy-id root@$A_MACHINE
Install Oracle 1.7 JDK RPM
Pivotal HD requires Oracle JDK 1.7 to be installed. You will need to download it using a browser, as Oracle requires you to accept their licensing agreement before downloading. You can download the 1.7 JDK from links on http://www.oracle.com/technetwork/java/javase/downloads/index.html. Download the 64 bit RPM to make installation easier. After downloading, you can scp (or perhaps pscp for Windows) the file to your new machine.After getting the file up to your machine, you can install the JDK with the following command in the directory you uploaded the RPM to:
yum -y localinstall jdk-7u*-linux-x64.rpm
Update alternatives
Unfortunately, Oracle's RPM doesn't properly update the alternatives links on CentOS. To update those links, issue the following command:alternatives --install /usr/bin/java java /usr/java/default/bin/java 1 \ --slave /usr/bin/jar jar /usr/java/default/bin/jar \ --slave /usr/bin/javac javac /usr/java/default/bin/javac \ --slave /usr/bin/javadoc javadoc /usr/java/default/bin/javadoc \ --slave /usr/bin/javaws javaws /usr/java/default/bin/javaws \ --slave /usr/bin/jcontrol jcontrol /usr/java/default/bin/jcontrol
Additional Steps for Virtual Machine Template
If you are creating a base template to use for Virtual based installs, then you will need to perform the following steps.Install VMware Tools
In vSphere (or Fusion or Workstation), execute the function to "Install VMware Tools". For Fusion, refer to the manual instructions at http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003417, for Workstation refer to http://pubs.vmware.com/workstation-10/index.jsp?topic=%2Fcom.vmware.ws.using.doc%2FGUID-012378D8-A995-4B78-AAD3-5A4223C4093E.html, and for ESX refer to http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1014294.Which ever method you use, you will need to execute the following commands to install VMware Tools in your CentOS machine:
mount /dev/cdrom /mnt cd /tmp tar zxvf /mnt/VMwareTools-*.tar.gz cd /vmware-tools-distrib ./vmware-install.pl umount /mnt rm -rf vmware-tools-distrib
Make Networking work after Cloning
If you are building a VM for cloning, you will want to execute the following commands to make networking work when the VM is started after a clone:sed -i "/UUID=.*/d" /etc/sysconfig/network-scripts/ifcfg-eth0 sed -i "/HWADDR=.*/d" /etc/sysconfig/network-scripts/ifcfg-eth0 rm /etc/udev/rules.d/70-persistent-net.rules
If you start your base image again for any reason after this step, you will need to execute the last line of the commands above as the referenced file is generated after every boot.
Clone
At this point you can clone the VM you built above as many times as you need. After starting up the cloned VM, you will want to edit the host name using the following commands:If you need to change the host name for the machine you are working on, you can do so with the following commands (either replace $NEWFQDN with the new fully qualified host name, or set this variable before executing the commands):
sed -i 's/HOSTNAME=.*/HOSTNAME=$NEWFQDN/' /etc/sysconfig/network hostname $NEWFQDN
If you are using avahi, you need to restart it to get the new host name picked up:
service avahi-daemon restart
Comments