Four-Layer Knowledge Model

Problem

Flat knowledge stores treat every entry as equally authoritative. A machine-inferred guess about retry behavior sits next to a human-ratified security constraint. At scale — dozens of agents, thousands of traces — confidence does not equal consensus. When a synthesizer proposes "agents should always retry 3 times" based on pattern data, but the business context says retries cause duplicate transactions, treating that proposal as truth is dangerous. Without layered separation, every synthesized insight constrains agent behavior with the same weight as organizational policy.

The consequences compound:

• Stale context pollutes search. Team standup notes from three weeks ago rank alongside active policies.
• No promotion path. Everything is either "in the vault" or not. There is no staging area for review.
• No decay semantics. Ephemeral working memory and permanent canon share the same lifecycle.
• Audit is impossible. You cannot answer "which entries were machine-generated vs. human-ratified?"

Solution

SOMA separates knowledge into four layers, each with distinct write permissions, decay rules, and semantic weight. Data flows upward through explicit gates — never automatically.

Layer	Name	Purpose	Semantic Weight	Expires
L1	Execution Archive	Raw traces, reconciled entries, decayed entries	historical	Never
L2	Team Working Memory	Team-scoped ephemeral context	contextual	14 days
L3	Emerging Knowledge	Machine-proposed insights with confidence scores	advisory	90 days
L4	Institutional Canon	Ratified organizational truth	mandatory	Never

L1 is the bedrock — everything starts there or ends there. L2 is scoped to teams and decays fast. L3 holds proposals that need human review. L4 is organizational law.

Architecture

Data flows upward through workers and governance gates. Decay flows downward — expired entries sink to L1, never deleted. The Policy Bridge reads all layers but routes queries by intent.

Code Examples

Writing to each layer

import { writeToLayer } from './layers';

// Harvester writes raw traces to L1
await writeToLayer(vault, 'archive', 'harvester', {
  type: 'execution',
  id: 'exec-invoice-agent-20260321-001',
  name: 'Invoice Agent: process batch',
  status: 'completed',
  agent_id: 'invoice-agent',
  tags: ['graph-inferred'],
});

// Team-context ingestion writes to L2
await writeToLayer(vault, 'working', 'team-context', {
  type: 'insight',
  id: 'standup-backend-20260321',
  name: 'Backend team standup notes',
  status: 'active',
  team_id: 'backend',       // Required for L2
  decay_at: '2026-04-04',   // Required for L2
  tags: ['team-context'],
});

// Synthesizer writes proposals to L3
await writeToLayer(vault, 'emerging', 'synthesizer', {
  type: 'policy',
  id: 'proposal-retry-limit',
  name: 'Agents should cap retries at 3',
  status: 'active',
  confidence: 0.82,
  evidence_links: ['exec-invoice-agent-001', 'exec-email-router-042', 'exec-invoice-agent-019'],
  tags: ['synthesized', 'cross-agent'],
});

// Governance API writes ratified entries to L4
await writeToLayer(vault, 'canon', 'governance', {
  type: 'policy',
  id: 'canon-retry-limit',
  name: 'Agents must cap retries at 3',
  status: 'enforcing',
  origin_l3_id: 'proposal-retry-limit',
  ratified_by: 'reviewer-jane',
  ratified_at: '2026-03-21T15:00:00.000Z',
  tags: ['ratified'],
});

Querying by layer

import { queryByLayer } from './layers';

// Get all mandatory policies (L4 canon)
const canon = await queryByLayer(vault, 'canon');
// Returns only L4 entries — index-level filtering, not full scan

// Get team working memory
const teamContext = await queryByLayer(vault, 'working', {
  filter: { team_id: 'backend' },
});

// Get pending proposals sorted by confidence
const proposals = await queryByLayer(vault, 'emerging', {
  filter: { status: 'active' },
  sort: 'confidence_desc',
});

Permission enforcement

// This will throw LayerPermissionError
await writeToLayer(vault, 'canon', 'synthesizer', {
  type: 'policy',
  id: 'sneaky-auto-promote',
  name: 'Trying to bypass governance',
  status: 'enforcing',
});
// Error: Worker 'synthesizer' cannot write to layer 'canon'

Rules and Invariants

Worker Write Permission Matrix

Worker	L1 (Archive)	L2 (Working)	L3 (Emerging)	L4 (Canon)
Harvester	WRITE	-	-	-
Reconciler	WRITE	-	-	-
Synthesizer	-	-	WRITE	-
Cartographer	-	-	WRITE	-
Team-Context	-	WRITE	-	-
Governance	-	-	-	WRITE
Policy Bridge	READ	READ	READ	READ

Writes to the wrong layer are rejected with LayerPermissionError.

Required Fields per Layer

Layer	Required Fields
L1	type, id, name, status, source_worker
L2	All L1 fields + team_id, decay_at
L3	All L1 fields + confidence, evidence_links
L4	All L1 fields + origin_l3_id, ratified_by, ratified_at

Decay Rules

Layer	Decays?	Default TTL	Decay Target
L1	Never	-	-
L2	Yes	14 days (configurable per team)	L1
L3	Yes (unless promoted/rejected)	90 days (configurable globally)	L1
L4	Never	-	-

Structural Invariants (tested with 200-op random sequences)

1. No entity exists in two layers simultaneously.
2. Every layered entity has a known source_worker.
3. L1 and L4 entries never have decay_at.
4. Every L2 entry has team_id and decay_at.
5. vault.update() rejects changes to the layer field — layer transitions only happen through governance promotion or decay.
6. Entities cannot change layers in-place. Promotion creates a new L4 entry; decay creates a new L1 entry.
7. Every entity on disk has a matching index entry, and vice versa.