When a host boots from Auto Deploy, it is very common to see following alarms triggered on Esxi host
These alarms are triggered because host booted in a disk less environment and there are no place where system can store logs etc.
In this post we will focus on how to fix these issues. This article is majorly focused on configuring/changing Esxi host swap and scratch partition configuration. We will start with system swap.
About System Swap
System swap is a memory reclamation process that can take advantage of unused memory resources across an entire system. In case of memory contention situation, system swap allows Esxi to reclaim certain parts of memory that is not used for virtual machines. The reclaimed memory is written to a storage location.
When swap is enabled, you have a tradeoff between the impact of reclaiming the memory from another process and the ability to assign the memory to a virtual machine that can use it. Since accessing the data from storage is slower than accessing data from memory, so we should be very careful with it determining where to store the swapped data so that performance impact is minimal.
The ESXi host determines automatically where the system swap should be stored and marks that as the Preferred swap file location. This decision can be aided by selecting certain options. If the ESXi host does not find a feasible option, the system swap is not activated. These options are:
Datastore – Allows the use of a specific datastore to store the system swap
Host Cache – Allows the use of part of the host cache
Preferred swap file location – Allows the use of the preferred location configured for the host
Note: We need minimum of 1 GB free space to configure swap.
How to configure system swap:
Swap can be configured via Web-Client,Host profile and Power-CLI. We will discuss Web-Client and host profile method here.
Configuring Swap from Web-Client
To configure/change system swap settings, login to vCenter Web Client and navigate to Host and Cluster. Select the Esxi host and Manage > Settings > System > System Swap. Click on Edit button to specify swap location.
Select “Enabled” check box to activate system swap and check mark the “use host cache” and “Can use datastore specified by host” option.
If you want to specify a particular datastore where swapped data will be stored, you can check mark “can use datastore” option and then select a datastore from drop down menu.
Configuring Swap via Host Profile
Edit your host profile and expand ‘General System Settings’ and select sub-profile “System Swap Settings“. Enable swap and select appropriate options as per your environment need. To specify a particular datastore to be used for storing swapped data, we need enable “Datastore Option” and provide datastore name.
If you are a Power-CLI fan and want to do some scripting to enable swap on all host then please check this article from Aaron Margeson. Also vBrown-Bag have mentioned some one liner Power-CLI script.
Scratch Partition Configuration
Scratch partition is nothing but a partition which is used to store vm-support bundle when at the time on troubleshooting a support bundle is generated. Although it is not mandatory to have a scratch partition, it is recommended to have it configured because it is useful for troubleshooting purposes.
During Esxi installation, the installer creates a 4GB VFAT scratch partition is created if it’s not present on another disk. When the host boot, the system tries to find a suitable partition on a local disk to create the scratch partition. If no scratch partition exists, then the support bundle is stored on host’s ramdisk.
In normal situation this doesn’t seems to be a big deal, but in memory contention situation, having a scratch partition starts to seems very important. Also if the support bundle is stored in ramdisk, they will disappear after host reboot.
A minimum of 5.2 GB of free space is required on the installation disk for the scratch partition to be created.
Before you start setting up the scratch location you need to make a decision which datastore you are going to use for this. You need to create a dedicated folder for each host so that hosts’ do not overwrite each others data.
You can create these folders either via doing a SSH to a host and then navigating to /vmfs/volumes/datastore and then use mkdir command to create folders for each host or you can do this from vSphere Web Client by browsing datastores.
Note: Datastore chosen for creating per/host folder should be visible to all ESXi hosts.
There are various methods by which we can create/configure scratch partition for Esxi hosts. Most commonly used are via Web Client, SSH command line and via Power-CLI. We will discuss all three method one by one.
Configuring scratch partition using vSphere Web Client
login to vCenter Web Client and navigate to Host and Cluster. Select the Esxi host and Manage > Settings > Advanced System Settings and type scratch in the search box.
Select the first option ‘ScratchConfig.ConfiguredScratchLocation‘ and edit this by clicking on the pencil button as shown below:
Enter the full path of he folder which you have created for your host. Remember to enter the path name using datastore UUID and not datastore name.
Reboot Esxi host for changes to take effect. Post host reboot verify that configuration is persistent.
Configuring scratch partition using Tech-Support Mode
The information about scratch partition configuration is written to the host’s /etc/vmware/locker.conf configuration file for use during the next boot.
Out of curiosity I just checked how locker.conf file looks and what info is stored there.
[root@esxi01:~] cat /etc/vmware/locker.conf
Now lets proceed with creating/configuring scratch partition using command line
Create a dedicated folder for your Esxi host
[root@esxi02:~] cd /vmfs/volumes/iSCSI-1/
[root@esxi02:/vmfs/volumes/591ac3ec-cc6af9a9-47c5-0050560346b9] mkdir -p Esxi02/scratch
[root@esxi02:/vmfs/volumes/591ac3ec-cc6af9a9-47c5-0050560346b9] cd Esxi02/scratch
Make a note of the full path to the folder created
Review the current scratch configuration
[root@esxi01:~] vim-cmd hostsvc/advopt/view ScratchConfig.ConfiguredScratchLocation
key = "ScratchConfig.ConfiguredScratchLocation",
value = "/vmfs/volumes/5916bead-baa2874b-367f-0050560346b9"