Scripted install of ESX 3.5

I am currently working with ESX 4 (vSphere) but wanted to document how I performed scripted installations with ESX 3.5. Unfortunately a lot has changed in ESX4 Update 1 requiring a lot of this script to be modified to accomplish the same goal. I’ll go over those details in the next post. 

There are many ways to install ESX but one of the fastest and easiest ways to keep your environment uniform is through a scripted install. This can be done via free UDA solutions, costly management solutions such as Altiris, or simply with a script and the installation media. I’ll cover option C. This script can be used to accept input variables that dynamically set the server’s hostname and IP address, setup all networking configurations as well as install Altiris and HP management agents. Simply comment out or delete the sections you don’t want.

This script is tailored for HP servers but should work with other brands as well, just make sure to change the partition and mgmt agents sections. The script is well commented so should be very self explanatory from section to section. It has served me well allowing me to build and rebuild ESX servers uniformly and quickly.

########### ESX 3.5 KICKSTART SCRIPT ##############

#Installation Method
url --url ftp://anonymous:rdp@x.x.x.x/esx35

# Regional Settings
keyboard us
lang en_US
langsupport --default en_US
timezone America/Chicago

# Installatition settings: no X windowing, no mouse, no firewall
skipx
mouse none
firewall --disabled

# root password (generate your own and paste below)
rootpw --iscrypted <$1$6…>

# Authconfig
auth --enableshadow --enablemd5

# BootLoader ( The user has to use grub by default )
bootloader --location=mbr

#Install or Upgrade then reboot
install
reboot

# Text Mode
text

# Network install type
# Create a default network for Virtual Machines
network --bootproto static --ip 0.0.0.0 --netmask 255.255.255.0 --gateway 0.0.0.0 --nameserver 0.0.0.0 --hostname ESX1 --addvmportgroup=0

#(experimental) auto-populated system fields from Altiris database
#network --bootproto static --ip=%NWSERVER% --gateway=%NWTREE% --netmask=%#NWCONTEXT% --nameserver= --hostname %#*"select replace([name],' ','') from computer #where computer_id={ID}"% --addvmportgroup=1 --vlanid=0

# Driver disks

# Load drivers

# ignoredisk to prevent installation on SAN
# This might NOT work for brands other than HP
# HP has its first disk on cciss/c0d0, LUNs are seen as sda, sdb, etc
ignoredisk –drives=sda,sdb,sdc,sdd,sde,sdf,sdg,sdh,sdi,sdj,sdk

# Bootloader options
bootloader --location=mbr --driveorder=cciss/c0d0 

# Authentication
auth --enableshadow --enablemd5

# Partitioning
#/, boot, & swap on c0d0
#/var, /tmp, /home, & vmkcore on c0d1
# %hddevice% is replaced with the detected storage device name by the
# vmesx.sh script executed on the target server.
# To specify specific or custom device names simply replace %hddevice%
# with specific device names ( Ex cciss/c0d0 ).
clearpart --all --drives=cciss/c0d0,cciss/c0d1 --initlabel
part /boot --fstype ext3 --size 250 --ondisk cciss/c0d0
part / --fstype ext3 --size 10240 --ondisk cciss/c0d0
# Swap partition = 2x service console RAM, max=800MB but disk is cheap. ;)
part swap --size 2048 --ondisk cciss/c0d0
part /var --fstype vmfs3 --size 4096 --ondisk cciss/c0d1
part /tmp --fstype ext3 --size 4096 --ondisk cciss/c0d1
part /home --fstype ext3 --size 2048 --grow --ondisk cciss/c0d1
#vmkCore partition = 100MB per VMware recommendation
part None --fstype vmkcore --size 100 --ondisk cciss/c0d1

# Loading network configuration from /tmp/networkconfig. This file
# is created in the %PRE section

# Network Configurations
%include /tmp/networkconfig

# VMWare License options
vmaccepteula
vmlicense --mode=server --server=27000@vc1.domain.com.com --edition=esxfull --features=vsmp

%vmlicense_text

%packages
@base

# +--------------------------------------+
# | Start ESX 3.5 install                |
# +--------------------------------------+

#+-----------------+
#| Begin %PRE      |
#+-----------------+

%pre
# In the pre section we read a parameter from the commandline
# Example commandline in the UDA
# append ip=dhcp ksdevice=eth0  load_ramdisk=1 initrd=initrd.esx301 network ks=http://x.x.x.249/kickstart/TST31.cfg ESXIP=49
# Important is the ESXIP=49 part. An ESX host that should get 192.168.1.49 as IP, will get ESXIP=49
# Because the names of my esx hosts is always:  vmesx001, vmesx002, vmesx003, ... vmesx049, I re-use the IP part for
# the dns name. Also, my vmotion interfaces will always have the same ending digit of the IP address, but just a
# different subnet

# I now read the commandline used to start the script. Each part is put in a variable. But I only need the ESXIP variable
# The = is used as a delimiter for the var

set -- `cat /proc/cmdline`
for I in $*; do case "$I" in *=*) eval $I;; esac; done

echo IP adres found $ESXIP

# Create /tmp/networkconfig file which will be read in a later section
cat << EOF >> /tmp/networkconfig
network --device eth0 --bootproto static --ip x.x.x.${ESXIP} --netmask 255.255.255.0 --gateway x.x.x.254 --nameserver y.y.y.203 --nameserver y.y.y.204  --hostname vmesx0${ESXIP}.domain.com --addvmportgroup=0
EOF

# Because the ESXIP variable is lost when switching fro, %PRE to the next section
# I write them to a file. The call-script.sh file is purely used to store the variables between sections.
# Later on I will call /tmp/esx-post-script.sh using the host IP address, followed by the vmotion IP

cat << EOF1 >> /call-script.sh
echo Now start /tmp/esx-post-script.sh
/tmp/esx-post-script.sh x.x.26.${ESXIP} x.x.28.${ESXIP}
EOF1
chmod a+x /call-script.sh

#+----------------------+
#| End PRE section      |
#+----------------------+

#+-----------------------------------+
#| Begin %POST --nochroot section    |
#+-----------------------------------+
# In the %PRE section I've written the vars to /call-script.sh. But when the %POST section starts,
# a new filesystem is mounted. And the file is lost. There is this special -nochroot section that
# enables both filesystems and enables me to copy the file from one filesystem to the next filesystem.

%post --nochroot
cp /call-script.sh /mnt/sysimage/call-script.sh

#+-----------------+
#| Begin %POST     |
#+-----------------+
# Post install
%post
# Transfer the Altiris agent along with its config files
mkdir /tmp/altiris
cd /tmp/altiris
ftp -n <<EOF2
open %#*"select tcp_addr from aclient_prop where computer_id=0"%
user anonymous rdp
cd /dslib/osoem/altiris
binary
prompt
mget altiris*.i386.bin
mget adlagent.conf.custom
mget adlagent.conf.default
exit
EOF2

AltirisConfDir=/opt/altiris/deployment/adlagent/conf
# Create script to configure ESX and install adlagent (called by rc.local)
# Using echos to thwart post section script variable and command substitution
echo '#!/bin/bash' >> ./hpinstall.sh
echo '# Script to configure ESX and install adlagent.  Called from rc.local.' >> ./hpinstall.sh

echo '# RDP install log file' >> ./hpinstall.sh
echo 'logfile=/root/install.rdp.log' >> ./hpinstall.sh
echo '# Create vmfs filesystem' >> ./hpinstall.sh
echo 'vmfsqueuedir="/etc/vmware/vmfs3queue"' >> ./hpinstall.sh
echo 'filecount=$(ls -1A /vmfs/volumes | wc -l)' >> ./hpinstall.sh
echo '# Check for existing vmfs volumes' >> ./hpinstall.sh
echo 'if [ $filecount -eq 0 ]; then' >> ./hpinstall.sh
echo '   # No current vmfs volumes' >> ./hpinstall.sh
echo '   # Check vmfs fs creation queue in case ESX is waiting to build on next boot' >> ./hpinstall.sh
echo '   if [[ -s $vmfsqueuedir ]]; then' >> ./hpinstall.sh
echo '      # Items in queue' >> ./hpinstall.sh
echo '      echo vmfs fs queue contains data - no vmfs created >>$logfile' >> ./hpinstall.sh
echo '   else' >> ./hpinstall.sh
echo '      # Nothing in queue' >> ./hpinstall.sh
echo '      # All clear to go ahead and create vmfs fs' >> ./hpinstall.sh
echo '      # Create vmfs fs' >> ./hpinstall.sh
echo '      vmfsdevice=`fdisk -l | grep %hddevice% | grep fb | cut -d" " -f1`' >> ./hpinstall.sh
echo '      partnum=${vmfsdevice:(-1)}' >> ./hpinstall.sh
echo '      vmfspart=`esxcfg-vmhbadevs | grep %hddevice% | cut -d" " -f1`' >> ./hpinstall.sh
echo '      echo "Creating vmfs fs on $vmfspart:$partnum" >>$logfile' >> ./hpinstall.sh
echo '      vmkfstools -C vmfs3 -S localvmfs $vmfspart:$partnum' >> ./hpinstall.sh
echo '   fi' >> ./hpinstall.sh
echo 'else' >> ./hpinstall.sh
echo '   # vmfs volumes exist' >> ./hpinstall.sh
echo '   echo vmfs fs volumes exist - no vmfs created >>$logfile' >> ./hpinstall.sh
echo 'fi' >> ./hpinstall.sh

cat >> ./hpinstall.sh << EOF1
# Firewall Configuration
# Enable adlagent  and file transfer ports
# You need to set a static port ("4300" in this example) for file transfer in
# the deployment console under Tools->Options->Global
esxcfg-firewall --openPort 402,tcp,out,adlagent
esxcfg-firewall --openPort 4300,tcp,out,adlagentFileTransfer

# Install Altiris Adlagent
cd /tmp/altiris
chmod +x altiris-adlagent*.bin
./altiris-adlagent*.i386.bin 1>>/root/install.rdp.log 2>>/root/install.rdp.log
# Install adlagent custom configuration
if [ -e adlagent.conf.custom ]; then
   mv $AltirisConfDir/adlagent.conf $AltirisConfDir/adlagnet.conf.bak
   cp -f adlagent.conf.custom $AltirisConfDir/adlagent.conf
elif [ -e adlagent.conf.default ]; then
   mv $AltirisConfDir/adlagent.conf $AltirisConfDir/adlagent.conf.bak
   sed -e "s/0.0.0.0/%#*"select tcp_addr from aclient_prop where computer_id=0"%/g" adlagent.conf.default > $AltirisConfDir/adlagent.conf
fi
# Reset adlagent to pick up config if necessary
/etc/init.d/adlagent stop
/etc/init.d/adlagent start

# Reset rc.local to original
mv -f /etc/rc.d/rc.local.sav /etc/rc.d/rc.local
EOF1

# make hpinstall.sh executable
chmod +x /tmp/altiris/hpinstall.sh

# save a copy of rc.local
cp /etc/rc.d/rc.local /etc/rc.d/rc.local.sav

# add hpinstall.sh to rc.local
cat >> /etc/rc.d/rc.local << EOF
cd /tmp/altiris
/tmp/altiris/hpinstall.sh
EOF

# +--------------------------------------+
# + Download script that is the same for +
# + each host                            +
# +--------------------------------------+

lwp-download http://x.x.x.249/scripts/esx-post-script.sh /tmp/esx-post-script.sh
chmod a+x /tmp/esx-post-script.sh

# +------------------------------------------+
# + Call script containing vars              +
# +------------------------------------------+
echo Now running /call-script.sh
/call-script.sh

########### ESX POST KICKSTART SCRIPT ##############
#!/bin/sh

#+--------------------------------------------------------------------------+
#| Just a test to show upon starting script that parameters have been received |
#| from %PRE sectie through /call-script.sh                               |
#+--------------------------------------------------------------------------+

#ESXHOSTIP=$1
#ESXVMOTIONIP=$2

#echo Found ESX Host IP = $1
#echo Found VMotion IP  = $2

# +--------------------+
# | Deploy ESX patches |
# +--------------------+

# download esx-autopatch.pl script
#lwp-download http://x.x.x.249/patches/3.0.1/esx-autopatch.pl /root/esx-autopatch.pl

# call esx-autopatch.pl script
#perl /root/esx-autopatch.pl

# +---------------------------------------------------------------------------+
# | Creation of /tmp/esxcfg.sh file. This files contains command that         |
# | can only be executed when the VMkernel is loaded                  |
# +---------------------------------------------------------------------------+

cat > /tmp/esxcfg.sh <<EOF1
#!/bin/sh

#-----Set Service Console Memory to 512M-----------------------------------------
#backup esx.conf and grub.conf
/bin/cp /etc/vmware/esx.conf /etc/vmware/esx.conf.bak
/bin/cp /boot/grub/grub.conf /boot/grub/grub.conf.bak
#editing esx.conf and grub.conf
/bin/sed -i -e 's/272/512/' /etc/vmware/esx.conf
/bin/sed -i -e 's/272M/512M/' /boot/grub/grub.conf
/bin/sed -i -e 's/277504/523264/' /boot/grub/grub.conf

# +---------------------------------------------------------------------------+
# | Create the Service Console                                                    |
# | vSwitch0 creation to assign Service Console                           |
# +---------------------------------------------------------------------------+

#Create and name VSwitch0
esxcfg-vswitch -a vSwitch0:32
esxcfg-vswitch -A "Service Console" vSwitch0
#Link vSwitch0 to vmnic0 (pNIC0)
esxcfg-vswitch -L vmnic0 vSwitch0
#Assign vswif interface and assign IP
esxcfg-vswif -a vswif0 -p "Service Console" -i 0.0.0.0 -n 255.255.255.0

# +---------------------------------------------------------------------------+
# | Create the Production0 vSwitch                                             |
# | vSwitch1 creation and NIC assignments                                  |
# +---------------------------------------------------------------------------+
#Create and name vSwitch1
esxcfg-vswitch -a vSwitch1:1016
esxcfg-vswitch -A Production0 vSwitch1
#Add pNICs 2, 3, 4, and 5 to vSwitch1
esxcfg-vswitch -L vSwitch1 vmnic1
#esxcfg-vswitch -L vSwitch1 vmnic3
#esxcfg-vswitch -L vSwitch1 vmnic4
#esxcfg-vswitch -L vSwitch1 vmnic5

# Restart vmware mgmt service for Virtual Center
service mgmt-vmware restart

# +---------------------------------------------------------------------------+
# | **Begin Altiris Sample Portion**                                               |
# +---------------------------------------------------------------------------+
# During auto-install a vSwitch0 is created, but I wanted a different naming
#esxcfg-vswif -d vswif0
#esxcfg-vswitch --del-pg='Service Console' vSwitch0
#esxcfg-vswitch -d vSwitch0
#
# Creation of the Service Console
#esxcfg-vswitch -a vsw-cos
# Connect physical nics vmnic0 & vmnic6 to vsw-cos
#esxcfg-vswitch -L vmnic0 vsw-cos
#esxcfg-vswitch -L vmnic6 vsw-cos
# Connect portgroups to vsw-cos
#esxcfg-vswitch --add-pg='Service Console' vsw-cos
# Next assign the IP address
# ===========================================================================
#esxcfg-vswif -a vswif0 -p 'Service Console' -i $ESXHOSTIP -n 255.255.255.0
# ===========================================================================
# +---------------------------------------------------------------------+
# | Creeating VMOTION kernelswitch                                    |
# +---------------------------------------------------------------------+
#
#esxcfg-vswitch -a vsw-vmotion
#esxcfg-vswitch -L vmnic2 vsw-vmotion
#esxcfg-vswitch -L vmnic8 vsw-vmotion
#esxcfg-vswitch --add-pg=vmotion vsw-vmotion
# Assign an IP address to the vmotion interface.
# This doesn't enable vmotion yet !!
# ===========================================================================
#esxcfg-vmknic -a "vmotion" -i $ESXVMOTIONIP -n 255.255.255.0
# ===========================================================================
# Setting vmkernel default gateway
#esxcfg-route x.x.x.254
#
# +-------------------------------------------------------------------+
# | Creating portgroups / VLANS                                        |
# | First create vsw-vms01 which will be used by all VMs     |
# +-------------------------------------------------------------------+
#
#esxcfg-vswitch -a vsw-vms01
#
#esxcfg-vswitch -L vmnic1 vsw-vms01
#esxcfg-vswitch -L vmnic3 vsw-vms01
#esxcfg-vswitch -L vmnic7 vsw-vms01
#esxcfg-vswitch -L vmnic9 vsw-vms01
#
# Create portgroups and add them to vSwitch vsw-vms01
#esxcfg-vswitch --add-pg=VLAN0019 vsw-vms01
#
#
# Now we connect the VLAN ID to a portgroup
#esxcfg-vswitch -v 19  -p VLAN0019 vsw-vms01
#
# +------------------------------------------------------------------+
# | **END Altiris Sample portion**                                      |
# +------------------------------------------------------------------+

# +------------------------------------------------------------------+
# | Firewall Configuration                                                |
# +------------------------------------------------------------------+

# Open for SSH client
esxcfg-firewall -e sshClient

# Open for SSH Server
esxcfg-firewall -e sshServer

# Open for ntp out
esxcfg-firewall -e ntpClient

# Open for SNMP
esxcfg-firewall -e snmpd

# Open for FlexLM out
esxcfg-firewall -e LicenseClient

# Open for CIM server services
esxcfg-firewall -e CIMSLP
esxcfg-firewall -e CIMHttpServer
esxcfg-firewall -e CIMHttpsServer

# Open for Virtual Center heartbeats
esxcfg-firewall -e vpxHeartbeats

# Open for FTP out
esxcfg-firewall -e ftpClient

# Open for Kerberos services outbound (should be handled by esxcfg-auth)
#esxcfg-firewall -o 464,tcp,out,KerberosPasswordChange
#esxcfg-firewall -o 88,tcp,out,KerberosClient
#esxcfg-firewall –o 749,tcp,out,KerberosAdm

# Open for AAM Client (enabled by vpxa client)
#esxcfg-firewall -e AAMClient

# Open for HPSIM
esxcfg-firewall -o 2381,tcp,in,HPSIM

# Restart firewall to enable changes
service firewall restart

# +------------------------------------------------------------------+
# | Active Directory authentication for SSH                      |
# +------------------------------------------------------------------+

# Configure Active Directory authentication
esxcfg-auth --enablead --addomain=domain.com --addc=domain.com

# Add a user to the local database, which is also in Active Directory
useradd domainuser1
useradd domainuser2
useradd domainuser3

# DNS configuration
echo nameserver 0.0.0.0 >> /etc/resolv.conf
echo nameserver 0.0.0.0 >> /etc/resolv.conf

# +------------------------------------------------------------------+
# | NTP configuration                                                       |
# +------------------------------------------------------------------+

# Backup ntpd.conf and step-tickers file
mv /etc/ntpd.conf /etc/ntpd.conf.bak
mv /etc/ntpd/step-tickers /etc/ntpd/step-tickers.bak

# Add Servers to step-tickers
echo "dc2.domain.com" > /etc/ntp/step-tickers
echo "dc1.domain.com" >> /etc/ntp/step-tickers

# create ntp.conf
echo "restrict 127.0.0.1" > /etc/ntp.conf
echo "restrict dc1.domain.com mask 255.255.255.255 nomodify notrap noquery" >> /etc/ntp.conf
echo "restrict dc2.domain.com mask 255.255.255.255 nomodify notrap noquery" >> /etc/ntp.conf
echo "server dc1.domain.com" >> /etc/ntp.conf
echo "server dc2.domain.com" >> /etc/ntp.conf
echo "driftfile /var/lib/ntp/drift" >> /etc/ntp.conf

# Service restart
service ntpd restart

# Make ntp start a boot time
chkconfig --level 345 ntpd on

# Sync hardware clock
hwclock --systohc

# +-----------------------------------------------------------------+
# | Scripted HP Insight Manager Agent install                   |
# | Download agent tar to local tmp dir                            |
# | To download, first open the firewall                            |
# +-----------------------------------------------------------------+
#
#cd /tmp
#/usr/sbin/esxcfg-firewall --allowOutgoing
#lwp-download http://0.0.0.0/repository/hpmgmt-7.9.1-vmware3x.tgz /tmp/hpmgmt-7.9.1-vmware3x.tgz
#lwp-download http://0.0.0.0/hpagent/esx3/hpmgmt.conf /tmp/hpmgmt.conf
#
# extract tar file
#tar -zxvf hpmgmt-7.9.1-vmware3x.tgz
#
# execute auto install
#cd /tmp/hpmgmt/791
#./installvm791.sh --silent --inputfile /tmp/hpmgmt.conf
#/usr/sbin/esxcfg-firewall --blockOutgoing
#
# Unload VMFS2 drivers from kernel to increase LUN speed
#mv /etc/init.d/vmware /etc/init.d/vmware.old
#sed -e "s/echo \"vmfs2 vmfs2\"/\#echo \"vmfs2 vmfs2\"/g" /etc/init.d/vmware.old > /etc/init.d/vmware
#chmod 744 /etc/init.d/vmware
#
# End of first script
#EOF1
#
# All of the above has been sent to /tmp/esxcfg.sh (not been executed yet)
# next step is to make /tmp/esxcfg.sh executable
#chmod +x /tmp/esxcfg.sh
#
# Backup of original rc.local file
#cp /etc/rc.d/rc.local /etc/rc.d/rc.local.bak
#
# edit rc.local to call esxcfg.sh
# and to make rc.local reset itself after calling
#cat >> /etc/rc.d/rc.local <<EOF
#cd /tmp
#/tmp/esxcfg.sh
#mv -f /etc/rc.d/rc.local.bak /etc/rc.d/rc.local
#EOF
#
##############################END OF KICKSTART SCRIPT#########################