# Adversarial Post-Optimization Verification

## When to Use

After any major Hermes optimization pass (skill pruning, config changes, prompt changes, cleanup, model routing changes). Run this BEFORE declaring success — it's designed to find regressions, not validate improvements.

## Methodology

Attempt to **disprove** every optimization, not validate it. For each change:

1. **State the claim** — what should have improved?
2. **Design a test that would prove it broken** — not one that would prove it works
3. **Run the test** — measure what breaks
4. **Categorize**: Verified (passed adversarial test), Failed (regression found), Partially Verified (minor issue), Unable to Verify (can't test from this context)

## Key Test Vectors

### Skills
- Disabled skills: try `skill_view()` on a disabled skill (v0.16.0 blocks it — use `read_file()` instead)
- Compacted skills: load via `skill_view()`, verify all expected sections (When to Use, Key Commands, Pitfalls)
- Conditional activation: load conditionally-activated skills — do they stall?

### Config
- All config values: verify via `yaml.safe_load()` not regex
- Delegation: verify `delegation.provider` and `delegation.model` are set
- Auxiliary slots: verify all 6 auxiliary slots route to the expected model
- Compression: verify threshold value

### RepoMap
- Generation speed: time the script on the primary codebase
- File coverage: verify Kotlin/source files are included (not just build/config)
- Large repos: test on a >2000-file codebase — expect timeout or graceful fallback
- Depth: verify `--depth` parameter works correctly

### Memory
- USER.md: verify size is below `user_char_limit`
- Write capability: verify writes work by checking if file is below limit
- Stale references: grep MEMORY.md for any deleted paths

### Broken References
- Symlinks: `find ~ -xtype l` — expect 0
- Config: grep for deleted paths
- Scripts: grep for deleted paths
- Cron: check jobs.json for deleted paths

### Profiles
- File completeness: verify SOUL.md + config.yaml + profile.yaml exist
- Content differentiation: verify profile-specific content (VRAM mentions in coding, delegate_task in cloud)

## Common Regressions

| Symptom | Likely Cause |
|---------|-------------|
| `skill_view()` fails on disabled skill | v0.16.0 behavior — use `read_file()` |
| Compacted skill grew back to full size | Background subagent overwrote compacted version |
| Duplicate skill directories | `mlops/` category contains stale skills not in disabled list |
| delegation.model shows wrong model | Config was overwritten by `/model` command |
| Broken symlinks | Paths deleted during cleanup but symlinks left behind |
| USER.md over limit | New writes blocked by character limit |
| RepoMap times out | Repo >2000 files — use `--depth 4` limit |
| state.db still large after VACUUM | WAL not checkpointed — run `PRAGMA wal_checkpoint(TRUNCATE)` |

## Real Example (from this codebase, 2026-06-18)

After a major Hermes audit (113 skills disabled, 5 compacted, config optimized):

- **Found**: Duplicate `llama-cpp` skill in `mlops/inference/` (36KB full version vs 2.6KB compacted)
- **Found**: `hermes-diagnostics` regrew from 3KB to 8KB (subagent overwrote)
- **Found**: 3 broken symlinks (openclaw-mem0.service, ~/backups, audit-fix-backups)
- **Found**: MEMORY.md had stale references to deleted directories
- **False alarm**: Delegation showed wrong model (regex parser bug — actual config was correct)
- **Verified**: All 5 compacted skills functionally equivalent with references/ intact
- **Verified**: 12/12 conditional activation skills accessible
- **Verified**: RepoMap 14x faster than brute-force `find`
- **Verified**: USER.md writes unblocked (46% of limit)