This article explains why device flattening fails on a host with HA configured, and provides a few workarounds for this issue.
If the host is part of an HA cluster, attempts to flatten the box via the Factory reset boot option fail due to /store partition UUID mismatch.
Shortly after the flatten action is committed (by explicit "FLATTEN" command at the prompt), the installation aborts with an error concerning UUID mismatch, such as the one below:
Specified nonexistent partition UUID=6d4d531f-9791-4259-bb63-c6193418de98 in partition command
Additional steps must be performed on the host before flattening.
This issue does not apply to USB stick flattening.
Device flattening fails on a host with HA configured because of the DRBD layer below the /store regular filesystem.
During flattening, the UUID of the partition read by the system is different from the UUID read when the system is booted normally.
As the kernel configuration stanza in /recovery/grub2/grub.cfg is created on the normally booted system, the UUID in the stanza does not match during flattening, causing the action to fail.
Example during normal boot
/dev/sda8: UUID="758d9f5f-8b27-4216-81d8-4e86293d71f4" TYPE="ext4"
Example during flattening
/dev/sda8: UUID="e4af0850bc885ce2" TYPE="drbd"
Currently, there are three known possible workarounds:
Create a USB stick and flatten via USB boot.
Prior to flattening, remove the drbd and reformat the /store partition, then update the grub configuration to match the new UUID.
Identify the DRBD UUID and manually change the kernel stanza in grub.cfg.
Create a USB stick and flatten via USB boot
1. Consult the instructions in this version 2013.2 document: Installing STRM Using a Bootable USB Flash-Drive PDF Document.
Prior to flattening: Remove the drbd and reformat the /store partition, then update the grub configuration to match the new UUID
1. Prior to booting, stop all services:
# service tomcat stop
# service hostcontext stop
# service hostservices stop
# service crond stop
# service syslog-ng stop
2. Unmount /store/tmp and /store:
# umount /store/tmp
# umount /store
3. Reformat /store:
# mkfs.ext4 -v /dev/sda8
4. Remount /store:
# mount -o noatime,nobarrier /dev/sda8 /store
5. Make a /store/tmp directory and remount:
# mkdir /store/tmp
# mount -o noatime,nobarrier /dev/sda5 /store/tmp
6. Run the recovery script to update the recovery partition:
# /opt/qradar/bin/recovery.py -u
This should correctly identify the new UUID and change the grub configuration accordingly.
7. Reboot and continue with the normal flattening procedure.
Identify DRBD UUID and manually change the kernel stanza in grub.cfg
1. Reboot into Factory reset and, at the prompt for flattening, escape to the shell by typing the hidden command "shell".
2. Execute the blkid command, and note the UUID of the drbd partition
3. Mount the recovery partition, and modify the grub.cfg kernel stanza so that the /store UUID matches.
4. Reboot and resume the flattening procedure.