# CachyOS Snapshot Restore — Limine Hook Failures & Btrfs Subvol Swap

**Applies to:** CachyOS / Arch with btrfs + snapper + Limine bootloader  
**EFI convention:** ESP mounted **directly at `/boot`** (not `/boot/efi`)  
**Trigger:** `snapper undochange` fails with `ERROR: FAT32 boot partition is not mounted at '/boot'` or `pre hook failed: /etc/boot/hooks/pre.d/10-limine-reset-enroll`

## Symptom: Limine pre-hook fails during snapper restore

```
ERROR: FAT32 boot partition is not mounted at '/boot'.
WARNING: pre hook failed (exit code 1): /etc/boot/hooks/pre.d/10-limine-reset-enroll
The file: /boot/limine.conf is not found.
```

**Root cause:** The hook (`initialize_header()` → `check_esp()`) requires a FAT32 partition mounted at `ESP_PATH` (default `/boot`). Common causes:

1. **fstab boot entry is commented out** — `#UUID=... /boot vfat ...` — the ESP never mounts
2. **ESP mounted at `/boot/efi`** instead of `/boot` (GRUB/systemd-boot layout)
3. **Rescue/overlay environment** — `/boot` isn't mounted because fstab is from the overlay, not the real root

**Quick fix (in current session):**
```bash
# Find the ESP
lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT | grep vfat

# Mount it at /boot
sudo mount /dev/nvme0n1p1 /boot

# Verify
ls /boot/limine.conf
```

**Permanent fix (edit fstab in the root subvol):**
```bash
# Uncomment boot entry in fstab
sudo sed -i 's/^#\(UUID=.* \/boot\)/\1/' /etc/fstab
```

## Symptom: Snapper says "subvolume is not a btrfs subvolume"

When booted into a snapshot (overlay root), `snapper -c root undochange <N>..0` fails because snapper needs direct btrfs ioctl access.

**Fix: Direct btrfs subvolume swap**

This technique replaces the default `@` subvolume with a writable snapshot of a specific numbered snapshot. Safe when the overlay root is active (`@` isn't in use as `/`).

```bash
# 1. Mount the btrfs top-level (subvolid=5)
sudo mkdir -p /mnt/btrfs
sudo mount -o subvolid=5 /dev/nvme0n1p2 /mnt/btrfs

# 2. Backup the current @ (keeps a rollback target)
sudo mv /mnt/btrfs/@ /mnt/btrfs/@.broken

# 3. Create new writable @ from a numbered snapshot
#    Snapshots live at @/.snapshots/<N>/snapshot under the old @ name
sudo btrfs subvolume snapshot /mnt/btrfs/@.broken/.snapshots/164/snapshot /mnt/btrfs/@

# 4. The snapshot inherits old fstab — re-apply any needed fixes
#    (the previous @.broken's fstab changes don't survive the snapshot)
sudo sed -i 's/^#\(UUID=F594-D99E.* \/boot\)/\1/' /mnt/btrfs/@/etc/fstab

# 5. Clean up the old broken @ later (after confirming boot works)
sudo btrfs subvolume delete /mnt/btrfs/@.broken
```

**After reboot:**
- The kernel cmdline has `rootflags=subvol=/@` — it auto-picks the new `@`
- ESP mounts via the fixed fstab entry
- Snapper history from the old `@` is NOT carried over (`.snapshots` directory was a nested btrfs subvolume in the original, which becomes an empty directory in the new snapshot)

## Pitfall: Fstab fixes don't survive subvolume swap

When you create a new `@` from a snapshot (`btrfs subvolume snapshot .../164/snapshot /mnt/btrfs/@`), the **new `@` inherits the fstab from the snapshot's original creation time**, not from the `@.broken` you just renamed.

If you previously fixed fstab (e.g. uncommented `/boot`) in the old `@`, **that fix is lost** — you must re-apply it in the new `@`:

```bash
# After creating the new @, always check and re-apply any fstab fixes:
sudo sed -i 's/^#\(UUID=F594-D99E.* \/boot\)/\1/' /mnt/btrfs/@/etc/fstab
sudo cat /mnt/btrfs/@/etc/fstab | grep boot
```

Common fixes that get lost:
- Boot (ESP) entry uncommented
- CIFS mount credentials path
- Custom mount options

**Always verify fstab after a subvolume swap before rebooting.**

## If the fstab /boot entry is missing entirely

Common on CachyOS installs using the wrong device in comments:
```bash
# Add it explicitly
echo "UUID=F594-D99E  /boot  vfat  rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro  0  2" | sudo tee -a /etc/fstab
```

## Fish shell SSH pitfalls

CachyOS uses fish shell by default. SSH commands with bash syntax fail:

| Bad (fish) | Good (bash wrapper) |
|------------|---------------------|
| `$?` | requires `bash -c` wrapper |
| `grep -E "(foo\|bar)"` | `grep -E "(foo|bar)"` inside bash -c |
| `&&` chaining with `$?` | `bash -c "cmd1 && cmd2"` |
| escaped paths with `\` | use `bash -c` or Python `subprocess` via execute_code |
| `2>&1` | works in fish but `> output 2>&1` syntax may differ |

**Best practice:** Wrap multi-step commands in `bash -c '...'` when SSHing into fish:
```bash
ssh user@host bash -c 'sudo command1 && sudo command2'
```

## Checking which snapshot you're booted into

```bash
cat /proc/cmdline | grep -o "subvol=/[^ ]*"
# Example: rootflags=subvol=/@/.snapshots/164/snapshot
```

## Listing snapshots (from rescue environment)

```bash
# Mount btrfs top-level
sudo mount -o subvolid=5 /dev/nvme0n1p2 /mnt/btrfs

# Read snapshot info
for i in /mnt/btrfs/@/.snapshots/*/; do
  n=$(basename "$i")
  desc=$(sudo grep -o '<description>[^<]*' "$i/info.xml" 2>/dev/null | sed 's/<description>//')
  [ -n "$desc" ] && echo "Snapshot $n: $desc"
done
```