🤠 Billy America — Self-Audit Dashboard

A. Memory Fragility

Problems: #1, #5, #11, #18

The agent's memory is structurally fragile. Context is ephemeral — the 200K context window creates an illusion of infinite memory. Billy works for hours, accumulates 150K+ tokens, feels like it "remembers" everything, and never writes to disk. Then compaction hits and everything evaporates.

• Two AI systems (main Billy + voice Billy) don't share state
• No single source of truth that's both durable AND current
• MEMORY.md not updated after events complete → stale info persists forever
• Credentials scattered across files, env vars, config — no registry

B. Zero Monitoring / Verification

Problems: #2, #3, #4, #6, #10, #13, #14

Nothing is monitored. Nothing is tested. Skills break silently. Config changes aren't validated. Data isn't verified. There are no automated checks, no CI, no alerting. Everything relies on Saint catching problems manually.

• 6/7 skills broken for weeks — nobody noticed
• No pre-flight config validation before restarts
• No skill health checks, no cron-based validation
• Post-call summary broken since OC 2.15 — still broken

C. LLM Behavioral Tendencies

Problems: #3, #7, #8, #9, #10, #12, #17

The LLM has predictable failure modes that documentation alone won't fix. These are inherent to how LLMs work — they require mechanical enforcement: crons, checklists, validation gates.

• Deferring action — "I'll do this later" feels safe, but Saint wants execution NOW
• Presenting guesses as facts — LLM confidence without verification
• Building instead of reusing — doesn't check TOOLS.md, reference/, skills first
• Batching instead of streaming — completionism vs progressive updates
• Ignoring standing orders — each session starts fresh, instructions don't carry forward

🤝 Council Consensus

• ✅ Unanimous: Enable QMD backend with session indexing
• ✅ Unanimous: Raise softThresholdTokens significantly (50K-100K range)
• ✅ Unanimous: Lower session sync delta thresholds
• ✅ Unanimous: Compaction summaries are fundamentally lossy — not a memory strategy
• ✅ Unanimous: Behavioral + architectural fixes needed — neither alone is sufficient

Compaction — Earlier Flush + More Reserve

Raise softThresholdTokens from 8K → 80K (implemented). Council recommended 50K-100K range.

{
  "mode": "safeguard",
  "reserveTokensFloor": 30000,
  "memoryFlush": {
    "enabled": true,
    "softThresholdTokens": 80000,
    "prompt": "CRITICAL: Write ALL important context to memory/YYYY-MM-DD.md NOW.",
    "systemPrompt": "You are about to lose context. Write EVERYTHING important to disk."
  }
}

QMD Backend — Session Indexing + Extra Paths

Swaps search engine under memory-core. BM25 + vectors + reranking. Fallback to SQLite if QMD fails.

{
  "backend": "qmd",
  "citations": "auto",
  "qmd": {
    "includeDefaultMemory": true,
    "sessions": { "enabled": true, "retentionDays": 30 },
    "update": { "interval": "5m", "debounceMs": 10000, "onBoot": true },
    "limits": { "maxResults": 10, "timeoutMs": 5000 },
    "paths": [
      { "name": "projects", "path": "projects", "pattern": "**/*.md" },
      { "name": "research", "path": "research", "pattern": "**/*.md" },
      { "name": "ideas", "path": "ideas", "pattern": "**/*.md" }
    ]
  }
}

Context Pruning — TTL-Based

Trim old tool results before LLM calls. Reduces cache-write costs on Anthropic.

{
  "mode": "cache-ttl",
  "ttl": "5m",
  "keepLastAssistants": 3
}

Session Sync — Lower Thresholds

Index sessions more frequently. Previous: 50KB/25msg. Now: 10KB/10msg.

{
  "watch": true,
  "sessions": { "deltaBytes": 10000, "deltaMessages": 10 }
}

Model Aliases — Clean Up Duplicates

Remove 4.5 entries to avoid ambiguity. Both opus-4.5 and opus-4.6 have alias "opus".

// REMOVE — superseded by 4.6 versions:
"anthropic/claude-opus-4-5": { "alias": "opus" }
"anthropic/claude-sonnet-4-5": { "alias": "sonnet" }
"openrouter/anthropic/claude-sonnet-4-5": { "alias": "or-sonnet" }

Currently Enabled (3)

🟢 Must Enable (High Value, Low Effort)

🟡 Should Enable (Medium Value, Some Setup)

🔧 Hooks to Enable

🔴 Custom Skills — Issues Found

📦 Missing Binaries

Three Failure Modes

4 Basic Config Fixes → Our Mapping

Advanced Tools

Multi-Agent Memory Architecture

• Layer 1: Private memory per agent (MEMORY.md + daily notes) — ✅ Active
• Layer 2: Shared reference files (symlinked _shared/ directory) — 📋 Not implemented
• Layer 3: QMD with shared paths (all agents search same docs) — ✅ Partially (QMD indexes projects/, research/)
• Layer 4: Coordination agent ("Chief of Staff") — 📋 Not implemented

✅ DONE — Implemented Changes

📋 PROPOSED — Awaiting Implementation

❌ UNFIXED — Known Broken