Full Stack Architecture
SOMA is one layer in a five-layer architecture for agent governance. Each layer operates at a different timescale and produces different outputs.
Two numbering systems are used in this documentation:
- Architecture stack (this page): L0 (SPE) → L1 (AgentFlow) → L2 (SOMA) → L3 (AICP)
- Vault layers (inside SOMA): L1 (Archive) → L2 (Working) → L3 (Emerging) → L4 (Canon)
These are separate hierarchies. The architecture stack describes system layers. The vault layers describe knowledge governance tiers within SOMA. When a doc says "L3 proposal," it means vault L3 (Emerging Knowledge), not architecture L3 (AICP).
The Five Layers
L0: SPE (Stateful Personal Entity)
Agent-local memory. Each agent (or personal AI assistant) maintains its own preferences, task history, and per-agent learning. SPE is the bottom of the stack — the individual agent's persistent self-knowledge that survives across sessions.
SPE feeds upward: its execution traces become the raw material that AgentFlow captures and SOMA synthesizes.
Status: Conceptual. The theoretical foundation exists (the SPE thesis defines the memory model), but no runtime implementation exists yet.
L1: AgentFlow
Execution observability. AgentFlow captures every agent run as an ExecutionGraph — a directed acyclic graph of steps, decisions, tool calls, and outcomes. On top of that graph, AgentFlow provides:
- Process mining — variant detection, conformance checking, bottleneck analysis
- Guards — runtime policy enforcement powered by
PolicySourceplugins - OTel tracing — OpenTelemetry spans for every graph node, compatible with Jaeger/Tempo/etc.
- Event emission —
ExecutionEventandPatternEventstreams that downstream systems (like SOMA) consume
AgentFlow is the foundation everything else builds on. Without execution graphs, there is nothing to synthesize, govern, or enforce.
Status: Production-ready. Core APIs are stable. Guards, process mining, and OTel integration are shipping.
L2: SOMA (Structured Organizational Memory Architecture)
Knowledge synthesis, governance, and intelligence. SOMA's worker cascade ingests AgentFlow traces and builds a four-layer knowledge vault:
- L1 Execution Archive — raw execution entities harvested from AgentFlow events
- L3 Emerging Knowledge — synthesizer-extracted insights, decisions, constraints, contradictions
- L4 Canon — promoted policies that feed back into AgentFlow guards via the Policy Bridge
The cascade (Harvester, Reconciler, Synthesizer, Cartographer) is proven with 6,870+ entities from production agents. Governance (L3→L4 promotion) is working with auto-promote and manual review.
SOMA includes two intelligence layers:
Operations Intelligence — per-run monitoring:
- Efficiency scoring (cost-per-node analysis, wasteful loop detection)
- Conformance drift detection (linear regression on agent scores)
- Run receipts (step-by-step execution summary)
- Guard explainability (structured explanation on every violation)
- Outcome assertions (opt-in post-pipeline verification)
Decision Intelligence — per-step and cross-agent analysis:
- Decision extraction (NormalizedDecision from any agent framework)
- Pattern signatures (comparable decision chains with repeat collapse)
- Cross-agent divergence (which step fails, what the successful agent does differently)
- Agent health briefings (status + intelligence + peer comparison)
- Decision replay (step-by-step visualization with reasoning)
Status: Production. Worker cascade, governance, ops intelligence, and decision intelligence are built and running. 19 CLI commands. Dashboard integration with Health Briefing and Decision Replay views.
L3: AICP (AI Control Plane)
Organizational policy authority. AICP implements the Propose → Authorize → Execute pattern:
- An agent proposes to execute (calls AICP preflight endpoint)
- AICP evaluates against SOMA vault policies, organizational constraints, budgets
- If authorized, the agent executes with guidance recorded in its trace
- SOMA ingests the trace (including AICP consultation), learns, improves
AICP is the layer that closes the operational loop — turning SOMA's analytical recommendations into recorded, traceable policy consultations. Without AICP, SOMA observes and recommends. With AICP, agents consult and SOMA traces the provenance.
Status: Proposal and design captured. Agent-agnostic HTTP preflight API designed. Not yet implemented.
Current State vs Target State
CURRENT STATE (SOMA analytical, no AICP)
════════════════════════════════════════
Agent A executes ──► Traces ──► SOMA Harvester ──► Vault L1
│
Agent B executes ──► Traces ──►──────────────────► Vault L1
│
Synthesizer analyzes
BOTH agents' data
│
▼
L3: "B should adopt
A's approach at step 2"
(RECOMMENDATION)
│
▼
Available via:
CLI, Dashboard,
Policy Bridge
Agents execute INDEPENDENTLY. SOMA observes post-hoc.
Recommendations sit in vault. No proof of influence.
TARGET STATE (SOMA + AICP operational loop)
═══════════════════════════════════════════
Agent A executes ──► Traces ──► SOMA Harvester ──► Vault L1
│
Synthesizer analyzes
│
▼
L3: "max-failure-rate = 20%"
(POLICY with evidence from A)
│
Policy Bridge exposes policy
│
▼
┌─────────────────────────────────────────┐
│ AICP PREFLIGHT │
│ │
Agent B ──────────► │ GET /api/aicp/preflight?agentId=B │
"I want │ │
to run" │ Evaluates: B's failure rate = 0.45 │
│ Policy: max-failure-rate = 0.20 │
│ Source: Agent A's data (47 traces) │
│ │
│ Response: proceed=true, warning: │
│ "Recommend A's approach at step 2" │
└───────────────────┬─────────────────────┘
│
▼
Agent B executes with guidance
Trace includes: aicp.consulted=true
│
▼
SOMA ingests B's trace
Sees AICP consultation
Traces provenance:
A's data → policy → AICP check → B's decision
(CAUSAL CHAIN with recorded proof)
Data Flow
┌─────────────────────────────────────────────────────────────────┐
│ AICP (future) │
│ Propose → Authorize → Execute │
│ Agent-agnostic preflight API │
│ Records consultations for SOMA to trace │
│ │ │
└────────────────────────────┼─────────────────────────────────────┘
│ reads policies from SOMA vault
▼
┌─────────────────────────────────────────────────────────────────┐
│ SOMA (built) │
│ │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ DECISION INTELLIGENCE │ │
│ │ Decision extraction · Pattern signatures · Divergence │ │
│ │ Health briefings · Decision replay · Recommendations │ │
│ ├────────────────────────────────────────────────────────────┤ │
│ │ OPERATIONS INTELLIGENCE │ │
│ │ Efficiency · Drift · Receipts · Guard explain · Assertions│ │
│ ├────────────────────────────────────────────────────────────┤ │
│ │ WORKER CASCADE │ │
│ │ Harvester → Reconciler → Synthesizer → Cartographer │ │
│ ├────────────────────────────────────────────────────────────┤ │
│ │ FOUR-LAYER VAULT │ │
│ │ L1 Archive → L2 Working (opt) → L3 Emerging → L4 Canon │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │ │
└────────────────────────────┼─────────────────────────────────────┘
│ policies + thresholds
▼
┌─────────────────────────────────────────────────────────────────┐
│ AgentFlow (built) │
│ GraphBuilder → ExecutionGraph → EventEmitter │
│ Guards enforce policies from SOMA + AICP │
│ Process mining · OTel tracing │
│ │ │
└────────────────────────────┼─────────────────────────────────────┘
│ execution events
▼
┌─────────────────────────────────────────────────────────────────┐
│ SPE (conceptual) │
│ Agent-local memory: preferences, task history, learning │
│ Each agent runs here; traces flow UP to AgentFlow │
└─────────────────────────────────────────────────────────────────┘
The primary data flow is bottom-up (agents produce traces that flow up through the stack) with top-down policy enforcement (AICP constraints and SOMA-derived policies flow down into guards).
Three Feedback Loops
Loop 1: Per-Run (milliseconds to seconds)
Agent runs. AgentFlow captures the execution graph. Ops Intel checks efficiency, computes drift score, and produces a run receipt. Guards enforce existing policies. Decision extraction captures the step-by-step chain.
Timescale: ms to s. Participants: AgentFlow, Ops Intel.
Loop 2: Cross-Run (hours to days)
Many execution graphs accumulate. SOMA's worker cascade processes them: Harvester ingests and extracts decisions, Synthesizer produces insights and detects divergence, Cartographer discovers archetypes. Guards get smarter. Future runs benefit from past analysis.
Timescale: hours to days. Participants: AgentFlow (event source), SOMA (learning + recommendations).
Loop 3: Organizational (weeks to months)
AICP reviews fleet-wide data — aggregate efficiency scores, drift trends, failure patterns across teams. Organizational policy adjusts: budgets change, compliance rules tighten, approval requirements shift. These constraints flow down into SOMA governance and agent guard enforcement.
Timescale: weeks to months. Participants: AICP (policy authority), SOMA (knowledge), AgentFlow (enforcement).
Status Summary
| Layer | Status | Key Milestone |
|---|---|---|
| SPE | Conceptual | Thesis published, no runtime |
| AgentFlow | Production | Guards, process mining, dashboard |
| SOMA | Production | 6,870+ entities, 19 CLI commands, decision intelligence |
| Ops Intel | Built (in SOMA) | Efficiency, drift, receipts, assertions |
| Decision Intel | Built (in SOMA) | Extraction, patterns, divergence, briefings |
| AICP | Proposed | Preflight API designed, not implemented |