[sheepdog] auto recovery after mount failure

[Chris Webb]

Dietmar Maurer <dietmar at proxmox.com> writes:

> normally people mount sheepdog data disks in fstab. That way the disks 
> get mounted at boot. For example, we have the following mounts:
> 
> /dev/sda ==> /
> /dev/sdb ==> /var/lib/sheepdog
> 
> The problem occurs when mounting the sheepdog data disk fails 
> at boot time (for example: damaged disk, admin triggered reboot).
> In  that case /var/lib/sheepdog is empty (and on /dev/sda).
> 
> The boot process continues, and 'sheep' simply creates a new 'farm' 
> in /var/lib/sheepdog. Even worse, It immediately start auto-recovery,
> which fills the root disk in short time.
> 
> Any idea how to prevent that. Maybe we should not create/initialize the 
> storage automatically at startup, man used a 'mksheepdogfs' to initialize the dir.
> That way the 'sheep' can check if the directory is initialized/mounted?

I handled this differently in my rc.startup: rather than mounting the
partitions with mount -a from /etc/fstab, I labelled them with sheep-FOO,
and did something like

  SHEEP=$(blkid -s LABEL | sed -n 's/: LABEL="\(sheep-[^"]*\)".*/ \1/p')
  PORT=7000; while read DEVICE LABEL; do
    mkdir -p "/mnt/$LABEL" \
      && mount -o user_xattr "$DEVICE" "/mnt/$LABEL" \
      && sheep -D -p $PORT "/mnt/$LABEL" \
      && let PORT++
  done <<< "$SHEEP"

so I only got sheep daemons for partitions that I'd correctly mounted.

Cheers,

Chris.