Operations Intelligence
Operations intelligence is the foundational layer that monitors how agents perform — cost efficiency, conformance drift, run receipts, and guard explanations. It provides the aggregate metrics that decision intelligence then drills into.
The Stack
┌─────────────────────────────────────────────────────────────────┐
│ DECISION INTELLIGENCE │
│ "Agent B fails at step 2 because it uses tool X" │
│ Requires: per-step decision data from operations layer │
├─────────────────────────────────────────────────────────────────┤
│ OPERATIONS INTELLIGENCE │
│ "Agent B has 60% failure rate, rising over 10 runs" │
│ Provides: efficiency, drift, receipts, guard explanations │
├─────────────────────────────────────────────────────────────────┤
│ FOUR-LAYER KNOWLEDGE VAULT │
│ L1 Archive → L2 Working (optional) → L3 Emerging → L4 Canon │
│ Provides: structured knowledge, governance, evidence chains │
├─────────────────────────────────────────────────────────────────┤
│ WORKER CASCADE │
│ Harvester → Reconciler → Synthesizer → Cartographer │
│ Provides: ingestion, dedup, synthesis, mapping │
└─────────────────────────────────────────────────────────────────┘
Features
Cost Efficiency Scoring
Analyzes token costs across agent executions to find wasteful patterns.
┌──────────────────────────────────────────────┐
│ SOMA EFFICIENCY — Cost Analysis │
│ │
│ Runs analyzed: 87 │
│ Data coverage: 42% │
│ │
│ Mean cost/node: 120 tokens │
│ Median cost/node: 95 tokens │
│ P95 cost/node: 340 tokens │
│ │
│ ⚠ Wasteful: retry loop in fetch-data (5x) │
└──────────────────────────────────────────────┘
What it detects:
- Per-run and per-node token cost analysis
- Wasteful retry loops (same tool called repeatedly with increasing cost)
- Data coverage percentage (how many nodes have cost metadata)
- Aggregate statistics (mean, median, P95 cost-per-node)
CLI: soma efficiency
API: GET /api/soma/efficiency
Library: getEfficiency(graphs: ExecutionGraph[]): EfficiencyReport
Conformance Drift Detection
Tracks agent conformance scores over time using linear regression to detect degradation trends.
┌──────────────────────────────────────────────┐
│ Conformance Trend: agent-alpha │
│ │
│ 1.0 ─ ● ● ● ● ● │
│ 0.9 ─ ● ●──●──●── (regression) │
│ 0.8 ─ ● │
│ 0.0 ───────────────────── │
│ │
│ Status: degrading │
│ Slope: -0.0032/run · R²: 0.45 │
│ 12 data points │
│ │
│ ⚠ ALERT: conformance declining │
└──────────────────────────────────────────────┘
How it works:
- Each
soma pipelinerun records a conformance score per agent (1 - failureRate) - Scores accumulate in
conformance-history.json - After 10+ data points, linear regression detects trends
- Status:
stable(R² < 0.3),degrading(negative slope, R² > 0.3),improving(positive slope, R² > 0.3) - Drift alerts are created as insight entities when agents degrade
CLI: soma drift [--agent <id>]
API: GET /api/soma/drift?agentId=<id>
Library: detectDrift(history: ConformanceHistory): DriftReport
Run Receipts
Structured step-by-step summary of an individual agent execution.
┌──────────────────────────────────────────────┐
│ === Run Receipt === │
│ Run: node_001 │
│ Agent: soma-cartographer │
│ Status: success │
│ Duration: 484ms │
│ │
│ # │ Step │ Status │ Duration │
│ 1 │ soma-cartographer │ ok │ 484ms │
│ 2 │ llm-call:cluster │ ok │ 484ms │
│ │
│ Total token cost: no cost data │
└──────────────────────────────────────────────┘
CLI: soma receipt --trace <file>
API: GET /api/traces/:filename/receipt
Library: toReceipt(graph: ExecutionGraph): RunReceipt
Guard Explainability
When a guard blocks or warns an agent, the explanation includes structured data about why:
┌──────────────────────────────────────────────┐
│ max-failure-rate [SOMA Policy] │
│ 0.75 → limit 0.3 │
│ Based on last 50 executions │
└──────────────────────────────────────────────┘
Source types:
- Static — hardcoded rules
- SOMA Policy — learned from vault L3/L4 entries
- Adaptive — adjusted based on agent behavior
- Assertion — outcome-based verification (from decision intelligence)
Outcome Assertions
Opt-in verification that runs after the pipeline completes. Assertions check whether the pipeline produced the expected results.
createSoma({
vaultDir: '.soma/vault',
assertions: [{
name: 'vault-has-agents',
verify: () => vault.list('agent').length > 0,
}],
}).run();
Failed assertions create insight entities with tags: ['assertion-failure'].
From Operations to Decisions
Operations intelligence tells you what is happening:
- Agent X has a 60% failure rate
- The trend is stable (not improving)
- Cost per node is 340 tokens at P95
Decision intelligence tells you why and what to do about it:
- Agent X fails at step 2 because it uses tool Y instead of tool Z
- The successful peer agent uses a different approach at that step
- Recommendation: adopt the peer's approach
- Agent X's decisions were guided by vault policies derived from Agent Y's data
Together they form a layered intelligence stack:
┌─────────────────────────────────────────────────────────────┐
│ │
│ AICP CONTROL PLANE (future layer) │
│ "Should this agent proceed?" │
│ Propose → Authorize → Execute · Recorded provenance │
│ Closes the loop: agents consult vault before acting │
│ │ │
│ reads from │
│ │ │
│ ┌──────────────────────┴──────────────────────────────┐ │
│ │ │ │
│ │ DECISION INTELLIGENCE (built) │ │
│ │ "Why, and what should change?" │ │
│ │ Decision extraction · Pattern signatures · │ │
│ │ Cross-agent divergence · Recommendations │ │
│ │ │ │ │
│ │ builds on │ │
│ │ │ │ │
│ │ OPERATIONS INTELLIGENCE (built — this page) │ │
│ │ "What is happening?" │ │
│ │ Efficiency · Drift · Receipts · Guard explanations │ │
│ │ │ │
│ │ ──────── SOMA ────────────────────────────────── │ │
│ │ │ │
│ └──────────────────────────────────────────────────────┘ │
│ │ │
│ stored in │
│ │ │
│ FOUR-LAYER KNOWLEDGE VAULT │
│ L1 Archive → L2 Working (optional) → L3 Emerging → L4 Canon · Policy Bridge │
│ The organizational memory that connects all agents │
│ │
└─────────────────────────────────────────────────────────────┘
SOMA provides the analytical layers (ops intelligence + decision intelligence). The AICP control plane (future) adds the operational loop where agents consult the vault before executing and consultations are recorded for causal provenance.