Decision Intelligence
SOMA doesn't just track whether agents succeed or fail — it understands what decisions they make, why they make them, and how those decisions compare across agents.
Every agent execution is a sequence of decisions: which tool to call, what parameters to pass, how to handle the result, what to do next. Decision intelligence captures this full chain — the what, the why (reasoning), and the outcome — for every execution, successful or failed.
This enables:
- Understanding successful agents — what decisions make agent A reliable? What's the pattern?
- Diagnosing failing agents — at which step does it fail, and what different choice could fix it?
- Cross-agent learning — agent A and B do similar tasks but make different choices at step 2. Which approach works better?
- Best practice extraction — the most successful agent's decision pattern becomes the benchmark
The Problem
A typical agent trace captures the outcome: "agent ran, it succeeded/failed." But that hides the real story — the sequence of 27 decisions the agent made during execution, each with reasoning and outcomes. Without decision-level data, you can't answer: "What did the agent decide? Why? What worked? What didn't?"
Most agent frameworks capture this data in session logs, but it gets lost during ingestion:
┌─────────────────────────────────────┐ ┌─────────────────────────────┐
│ SESSION TRACE (raw data) │ │ VAULT ENTITY (what SOMA │
│ │ │ used to keep) │
│ 60 events │ │ │
│ 27 tool calls (decisions) ────────▶ │ 1 entity │
│ 28 reasoning blocks │ │ status: completed/failed │
│ 27 outcomes per step │ │ duration: total only │
│ per-step durations │ │ nodeCount: 2 │
│ tool names + arguments │ │ variant: 1 string │
│ error messages per step │ │ │
│ │ │ LOST: individual tool │
│ │ │ calls, reasoning, per-step │
│ │ │ outcomes, errors │
└─────────────────────────────────────┘ └─────────────────────────────┘
27 decisions with reasoning compressed into 1 status field. SOMA could say "agent fails 60%" but never explain why — because the evidence was dropped.
Data Flow
Decision intelligence adds a new extraction step to the Harvester, preserving per-step data through the entire pipeline:
┌─────────────────────────────────────┐
│ AGENT EXECUTION │
│ (any framework: OpenClaw, LangChain,│
│ AutoGen, custom) │
└──────────────┬──────────────────────┘
│
trace file (JSON/JSONL/OTel)
│
▼
┌──────────────────────────────────────────────────────────────────────┐
│ HARVESTER │
│ │
│ ┌─────────────┐ ┌──────────────────┐ ┌────────────────────┐ │
│ │ Adapter │ │ Decision │ │ Vault Write │ │
│ │ (format- │───▶│ Extraction │───▶│ │ │
│ │ specific) │ │ │ │ execution entity + │ │
│ │ │ │ toolCall → action │ │ decisions[] │ │
│ │ OpenClaw │ │ thinking → why │ │ decisionPattern │ │
│ │ LangChain │ │ toolResult → out │ │ │ │
│ │ OTel │ │ duration │ │ │ │
│ │ Generic │ │ │ │ │ │
│ └─────────────┘ └──────────────────┘ └────────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────────┐
│ RECONCILER → SYNTHESIZER │
│ │
│ ┌─────────────────┐ ┌────────────────────┐ ┌──────────────┐ │
│ │ Pattern │ │ Divergence │ │ L3 Insight │ │
│ │ Signatures │───▶│ Detection │───▶│ Creation │ │
│ │ │ │ │ │ │ │
│ │ exec:tool_a → │ │ Agent A: 95% │ │ "B diverges │ │
│ │ process[5] → │ │ Agent B: 40% │ │ from A at │ │
│ │ write:dir[5] │ │ │ │ step 2" │ │
│ │ │ │ Divergence at │ │ │ │
│ │ │ │ step 2: different │ │ confidence: │ │
│ │ │ │ tool choice │ │ high │ │
│ └─────────────────┘ └────────────────────┘ └──────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────────┐
│ DASHBOARD / CLI │
│ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ AGENT HEALTH BRIEFING │ │
│ │ │ │
│ │ STATUS: ⛔ CRITICAL — 60% failure rate (32/53 runs) │ │
│ │ │ │
│ │ ┌─ WHY IS IT FAILING? ──────────────────────────────────────┐ │ │
│ │ │ • Divergence: uses exec:node at step 2 │ │ │
│ │ │ (peer uses exec:tool_a with 95% success) │ │ │
│ │ │ • Constraint: "digest agents unreliable" │ │ │
│ │ └───────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ ┌─ WHAT SHOULD YOU DO? ─────────────────────────────────────┐ │ │
│ │ │ • Policy: "≥50% failure = immediate intervention" │ │ │
│ │ │ • Decision: benchmark against successful peer │ │ │
│ │ └───────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ ┌─ PEER COMPARISON ────────────────────────────────────────┐ │ │
│ │ │ agent-a ████████████████████░░ 95.5% │ │ │
│ │ │ agent-b ████████████████░░░░░░ 83.3% │ │ │
│ │ │ agent-c ████████░░░░░░░░░░░░░░ 39.6% ◀ │ │ │
│ │ └──────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ ┌─ DECISION REPLAY (last run) ─────────────────────────────┐ │ │
│ │ │ 1. ✔ exec:tool_a OK 1.2s │ │ │
│ │ │ 2. ✘ exec:node FAIL 30s (timeout) │ │ │
│ │ │ 3. ○ write:output SKIP — │ │ │
│ │ └──────────────────────────────────────────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────────┘
NormalizedDecision
The agent-agnostic representation of a single decision:
NormalizedDecision {
action: "exec:himalaya" // what was decided (enriched with tool context)
reasoning: "Fetch emails..." // why (from thinking/reasoning blocks)
tool: "exec" // tool used
outcome: "ok" // result
output: "5 emails returned" // output summary
durationMs: 1200 // time taken
}
This works across frameworks — each adapter knows how to extract decisions from its format:
| Framework | Decision = | Reasoning = | Outcome = | Reasoning Depth |
|---|---|---|---|---|
| Claude/OpenClaw | toolCall | thinking block | toolResult | Full — captures agent's reasoning before each tool call |
| LangChain | tool run | parent chain name | run outputs | Partial — only records which chain invoked the tool |
| OpenTelemetry | CLIENT span | span attributes | span status | Varies — depends on instrumentation quality |
| AgentFlow JSON | tool node | (not available) | node status | None — node metadata doesn't include reasoning |
The reasoning field on NormalizedDecision is optional by design. SOMA's decision intelligence works for all frameworks (actions, outcomes, patterns, divergence), but the why behind each decision is only available from frameworks that capture agent reasoning (currently Claude/OpenClaw sessions with thinking blocks).
Pattern Signatures
Decision chains are reduced to comparable pattern signatures:
exec:himalaya → process:poll[2] → exec:himalaya[6] → write:inbox[5] → exec:himalaya[5]
Normalization rules:
- Action names enriched with tool context (
exec→exec:himalaya) - Consecutive repeated actions collapsed (
write, write, write→write[3]) - Numeric suffixes stripped for comparison
Cross-Agent Divergence
When two agents perform similar tasks but have different success rates, SOMA compares their decision patterns to find the divergence point:
Agent A (95% success): exec:himalaya → process → write:inbox → exec:himalaya
Agent B (40% success): exec:node → exec:spawn → write:file
DIVERGENCE at step 1:
A uses: exec:himalaya (direct IMAP)
B uses: exec:node (subprocess spawn → timeouts)
This creates L3 divergence insights with evidence — actionable recommendations, not just statistics.
CLI Commands
# Show decision chain for a specific trace
soma decisions --trace <file>
# Compare decision patterns across agents
soma divergence
# Full health briefing for an agent
soma briefing --agent <id>
Dashboard Views
Agent Health Briefing
A single-page operational view per agent combining:
- Status: critical/degraded/healthy with failure rate
- Why it fails: constraints, divergence insights, contradictions
- What to do: triggered policies, decisions, recommendations
- Peer comparison: agents ranked by success rate
- Drift trend: conformance score over time
Decision Replay
Step-by-step visualization of a single execution's decision chain:
- Each step shows action, tool, outcome, duration
- Click to expand reasoning and output
- Failed steps highlighted in red
- Pattern signature displayed at bottom
Cross-Agent Intelligence
SOMA's vault contains rich cross-agent knowledge — 141 insights reference 2+ agents, 95 decisions cite multi-agent evidence. The synthesizer analyzes patterns across all agents and produces actionable recommendations.
What SOMA Does Today (Analytical)
SOMA observes all agents' traces and compares their behavior post-hoc:
┌──────────────────────────────────────────────────────────────────┐
│ SOMA: CROSS-AGENT ANALYSIS (working today) │
│ │
│ Agent A executes repeatedly │
│ │ │
│ ▼ │
│ Harvester ingests A's traces → L1 Archive │
│ │ │
│ ▼ │
│ Agent B executes repeatedly (independently) │
│ │ │
│ ▼ │
│ Harvester ingests B's traces → L1 Archive │
│ │ │
│ ▼ │
│ Synthesizer analyzes BOTH agents' data: │
│ "A succeeds 95% using exec:himalaya at step 2. │
│ B fails 60% using exec:node at step 2. │
│ Same task, different tool choice → different outcome." │
│ │ │
│ ▼ │
│ L3 DIVERGENCE INSIGHT: │
│ "B should adopt A's approach at step 2" │
│ Evidence: A's 47 traces, B's 32 traces │
│ │ │
│ ▼ │
│ Available via: soma divergence, soma briefing, │
│ Policy Bridge, Dashboard Health Briefing │
└──────────────────────────────────────────────────────────────────┘
SOMA tells you: "Agent B does the same job as Agent A but makes a different choice at step 2, and A's approach works better." This is a recommendation based on evidence, not a causal chain.
The vault holds this knowledge:
- L1 Archive: raw traces from all agents
- L3 Emerging: 141 synthesized insights cross-referencing agents
- L4 Canon: ratified policies derived from multi-agent evidence
- Policy Bridge: makes vault knowledge queryable for any system
What AICP Adds (Operational — Future Layer)
For agents to actually consult the vault before executing and have that consultation recorded, a separate control plane layer is needed: the AI Control Plane (AICP).
┌──────────────────────────────────────────────────────────────────┐
│ AICP: PROPOSE → AUTHORIZE → EXECUTE (future) │
│ │
│ Agent B wants to execute │
│ │ │
│ ▼ │
│ AICP checks SOMA vault via Policy Bridge: │
│ "B's failure rate 0.60 exceeds threshold 0.20. │
│ Policy derived from Agent A's data." │
│ │ │
│ ▼ │
│ AICP decision: WARN (proceed with guidance) │
│ "Recommend adopting A's approach at step 2" │
│ │ │
│ ▼ │
│ Consultation RECORDED in B's trace │
│ │ │
│ ▼ │
│ Agent B executes (with or without following guidance) │
│ │ │
│ ▼ │
│ SOMA ingests B's trace → sees AICP consultation │
│ → can now trace: A's data → policy → AICP check → B's decision │
│ → TRUE CAUSAL CHAIN with recorded provenance │
└──────────────────────────────────────────────────────────────────┘
The key difference:
- SOMA (today): "B should use A's approach" — analytical recommendation
- AICP (future): "B was told to use A's approach, and here's whether it did" — recorded causality
Business Scenarios
These scenarios show what SOMA can analyze today, and how AICP would close the loop:
Product → Marketing: feature adoption patterns
Product Agent traces (ingested by SOMA):
- Enterprise segment shows 3x adoption for collaboration features
- Evidence: 47 traces over 6 weeks
Marketing Agent traces (ingested by SOMA):
- Sends generic campaigns to all 50,000 customers
- Low engagement rate on collaboration messaging
SOMA ANALYSIS (works today):
"Product data shows enterprise segment has highest adoption.
Marketing campaigns don't segment by adoption patterns.
RECOMMENDATION: target enterprise for collaboration features."
Available via: soma divergence, Policy Bridge, Dashboard
WITH AICP (future):
Marketing agent would consult vault before campaign:
→ Receives product insight about enterprise segment
→ Records consultation in trace
→ Targets 2,000 enterprise customers instead of 50,000
→ SOMA traces: product data → insight → AICP check → campaign targeting
Support → Product: ticket patterns reveal priorities
Support Agent traces (ingested by SOMA):
- 42% of tickets reference batch-export capability
- Constraint: current export limited to 1,000 rows
- Evidence: 847 traces, 356 mention export
Product Agent traces (ingested by SOMA):
- Sprint planning doesn't reference support ticket patterns
- Batch-export sitting in backlog at low priority
SOMA ANALYSIS (works today):
"Support data shows batch-export is top customer pain point.
Product backlog has it at low priority.
RECOMMENDATION: reprioritize based on support evidence."
WITH AICP (future):
Product agent would consult vault before sprint planning:
→ Receives support constraint about export limitation
→ Records consultation in trace
→ Moves batch-export to current sprint
→ SOMA traces: support tickets → constraint → AICP check → sprint change
The Two Layers
SOMA (analytical — available today) AICP (operational — future)
═══════════════════════════════════ ════════════════════════════
Observes all agents' traces Agents consult vault before acting
Compares patterns across agents Records consultations in traces
Identifies divergence points Enables true causal provenance
Produces recommendations Closes the feedback loop
Post-hoc analysis Real-time authorization
"B should do what A does" "B was told to do what A does,
and here's whether it did"
No direct communication between agents in either layer. SOMA uses the vault as shared analytical memory. AICP would use the vault as shared policy authority.
Architecture
Decision intelligence extends the existing four-layer worker cascade:
Harvester (enhanced)
├── Ingests traces (existing)
├── Extracts NormalizedDecision[] via adapter (NEW)
└── Stores decisions + decisionPattern on execution entities (NEW)
Synthesizer (enhanced)
├── synthesize() — extract insights from entities (existing)
├── synthesizeL3() — create L3 proposals (existing)
├── synthesizeDecisions() — cluster by decision type (existing)
├── synthesizeDecisionDivergence() — compare across agents (NEW)
└── synthesizeCausalChains() — trace cross-agent knowledge flow (NEW)
Guard System (enhanced)
├── checkPolicyViolations() with policySource (existing, now wired)
└── Emits NormalizedDecision for each guard check (NEW)
No new workers are needed. The four-layer vault, Policy Bridge, and guard system provide the cross-agent coordination. The adapter system handles format differences so SOMA remains agent-agnostic.