VMware vCenter Server Appliance VCSA filesystem is damaged

By cpwpadmin|October 05th, 2015

VMware vCenter Server Appliance VCSA filesystem is damaged

Although you´ve done as much as you can do (or afford) to avoid a virtual machine crash, it can always happen. Sometimes you´re storage system goes down, sometime your HBA breaks or the fabric in between. Many misconfigurations lead to a non-redundant situation you find out about when it goes down. Therefore you should run an vSphere Health Analyzer Test to make sure.

If your VMware vCenter Server Appliance crashes and the VCSA filesystem is affected (especially VCSA 6, VMware vSphere 6) , it might happen that the system is not recovering anymore. You end up with a nice console message telling you that the filesystem is corrupt and you can try to recover it manually. You need to know that VCSA 6 is using a LVM (logical volume manager), so for most system administrators with little linux experience it can be pretty hard to troubleshoot.

You can get a better understanding when reading Cormac Hogan’s Post about fsck on VCSA 6.0 and the lvm partitions.

fsck failed.  Please repair manually and reboot.  The rootfile system is currently mounted read-only.  To remount it read-write do:

 bash# mount -n -o remount,rw 

Actually that is not so simple when using VCSA as the shell (bash) is not enabled by default and you don’t have a chance to do so without changing the grub boot entry.

VCSA Grub Entry

Within the grub boot menu you need to press p to get into the boot options, using vmware as the default password or your own password configured for the root account.

VCSA grub password

Select VMware vCenter Server Appliance and hit e to edit the grub entry.

VCSA grub bootmenu

Select the second entry that starts with kernel and hit e again to edit that.

VCSA Bootmenu edit kernel entry

Change the line and append init=/bin/bash and hit enter to enable the bash shell after the OS start.

VCSA grub boot bash

Press b to boot the changed grub entry

bash prompt to fix VCSA filesystem

You should see a command prompt to enter 

mount -n -o remount,rw

fsck -f -c -y

That starts fixing all filesystem errors. After a while you should get some messages about VCSA filesystem issues you need to confirm to fix them. Therefore it always makes sense to create a VM snapshot before starting the filesystem fix.

When all is done, the command reboot is all you need and VCSA should start without any issues.

You can read more about VMware vCenter Server Appliance (VCSA) filesystem issues here:

vCenter Appliance fsck failed 

fsck of vCenter Server Appliance 6.0 partitions

Share This Story, Choose Your Platform!


Related Posts

Log in

Forgot your password?
Don’t have an account? Register.