Sibyl — Rust Memory System

Self-hosted, local-first memory for LLM agents. 3-tier architecture with semantic search, self-learning, scheduled maintenance, and zero external dependencies.

---

Architecture

┌──────────────────────────────────────────────────────┐
│                     CLI (sibyl)                       │
│   status | search | entity | maintain | learn | heal │
│   dedup | journal | serve | sleep                    │
└──────────┬───────────────────────────┬───────────────┘
           │                           │
     ┌─────▼─────┐              ┌──────▼──────┐
     │ REST API  │              │  MCP Stdio  │
     │  :9090    │              │  :stdio     │
     └─────┬─────┘              └──────┬──────┘
           │                           │
           └──────────┬────────────────┘
                      ▼
        ┌─────────────────────────┐
        │     sibyl_store         │
        │  (SQLite, WAL mode)     │
        ├─────────────────────────┤
        │  entities               │
        │  relations              │
        │  journal_events         │
        │  user_memory            │
        │  working_memory         │
        │  entity_vectors (F32)   │
        │  FTS5 indexes           │
        └─────────────────────────┘

Tiers

Tier	Table	Purpose	TTL	Priority
Reference Docs	entities + reference_docs	FTS-indexed descriptions of services, repos, patterns, users	∞	+0.3 rerank
User Memory	user_memory	Key/value preference store — language, risk mode, conventions	Configurable	+0.5 rerank
Working Memory	working_memory	Session-scoped slots — agenda, context, last_turn	Configurable	Auto-set by Decider

Rust Workspace (9 crates)

Crate	Purpose
sibyl-core	Types, newtypes (EntityId, RelationId, TenantId), error
sibyl-store	SQLite schema, CRUD for all tables, WAL management
sibyl-search	FTS5, hybrid RRF, vector (2560-dim), graph traverse, temporal
sibyl-pipeline	Learn (self-learning), Miner (entity extraction), Maintain (vectors/FTS/archive), Heal (dedup/corruption/prune)
sibyl-api	Axum REST server — all endpoints
sibyl-mcp	JSON-RPC stdio server — MCP protocol
sibyl-bin	CLI + cron chains
sibyl-sleep	NREM compression (nightly cluster + compress)
sibyl-fuzz	Fuzz targets

Quick Start

# Build (production)
cargo build --release -p sibyl

# Run API server
sibyl serve --api

# Run MCP (Hermes integration)
sibyl serve --mcp --stdio

# CLI
sibyl status
sibyl search "database version"
sibyl search --vector "deployment infrastructure"
sibyl entity --name "traefik" --category "infra:service"
sibyl maintain      # daily maintenance
sibyl heal          # health checks
sibyl learn         # self-learning pass

REST API

GET    /health
GET    /api/v1/search?query=&fts=&vector=&category=
GET    /api/v1/traverse?entity_id=&depth=
GET    /api/v1/temporal/{recent,stale,growth,journal}
GET    /api/v1/entities
POST   /api/v1/entities
DELETE /api/v1/entities/:id
POST   /api/v1/maintain
POST   /api/v1/heal
GET    /api/v1/user-memory?key=
POST   /api/v1/user-memory
POST   /api/v1/wm
GET    /api/v1/wm?session_id=&slot=
POST   /api/v1/wm/gc
POST   /api/v1/wm/compress

Working Memory (Decider Integration)

Every post_llm_call from the Decider plugin auto-sets 3 WM slots:

decider:{session_id} → agenda      (priority 80, TTL 60m)
decider:{session_id} → context     (priority 70, TTL 120m)
decider:{session_id} → last_turn   (priority 50, TTL 30m)

Fire-and-forget via daemon thread — zero latency on the hot path.

Cron Chain

03:00 daily:
  parallel: learn + heal + maintain + memos-sync
  sequential: NREM compress

Deployment

# Systemd service
sibyl serve --api           # :9090
sibyl serve --mcp --stdio   # MCP for Hermes

# Timer units
sibyl-journal.timer         # every 5min
morning.timer               # 03:00 daily
noon.timer                  # 12:00 daily
weekly-maintenance.timer    # Sunday 03:00

Migration from Python

Sibyl started as Python (/opt/sibyl-local/). The Rust rewrite replaces all core services:

Service	Python	Rust	Status
API server	sibyl-api (:8648)	sibyl serve --api (:9090)	✅
MCP server	sibyl_memory_mcp	sibyl serve --mcp --stdio	✅
Journal plugin	sibyl-journal	sibyl journal	✅
CLI	sibyl (Python)	sibyl (Rust)	✅
Maintenance	Python pipeline	sibyl maintain	✅
Health	scripts/heal.sh	sibyl heal	✅
NREM	nrem-compress.sh	sibyl sleep nrem	✅

License

MIT