Decision Extraction

Problem

Agents make decisions constantly — which tool to call, whether to retry, when to delegate to a subagent, which branch to take. These decisions are the most valuable signal for organizational learning. But nobody tracks them. Execution traces capture what happened (tool calls, durations, outcomes) but not what was decided (the choice, the alternatives not taken, the implicit reasoning).

Without decision tracking:

• Pattern blindness. You cannot answer "do all agents prefer tool X over tool Y?" because the choice is implicit in the trace, not explicit in the data.
• No behavioral analysis. Retry patterns, delegation patterns, failure responses — all invisible.
• Adapter burden. Requiring every framework to emit structured decision events means it never happens. Adoption friction kills the feature.

Solution

SOMA infers decisions from ExecutionGraph structure — zero configuration required. Any framework that produces a graph with nodes and edges gets decision extraction automatically. The Harvester walks the graph, identifies five structural patterns, and writes decision entities to L1.

The key insight: decisions are already encoded in graph topology. A tool node means the agent chose that tool. A branched edge means the agent selected one path over alternatives. A retried edge means the agent decided to retry. The structure is the decision.

For frameworks that want richer data, SOMA also accepts optional explicit decision trace events with DecisionTraceData. But the default path — graph inference — requires nothing from the framework.

Architecture

Code Examples

Extracting decisions from a graph

import { extractDecisionsFromGraph } from './decision-extractor';

// A sample ExecutionGraph from an agent run
const graph = {
  id: 'exec-invoice-agent-20260321-001',
  agent_id: 'invoice-agent',
  nodes: [
    { id: 'n1', type: 'tool', name: 'fetch-data', status: 'completed',
      metadata: { duration_ms: 1200 } },
    { id: 'n2', type: 'tool', name: 'transform-payload', status: 'completed',
      metadata: { duration_ms: 340 } },
    { id: 'n3', type: 'subagent', name: 'validator-bot', status: 'completed',
      metadata: { parent_agent: 'invoice-agent' } },
    { id: 'n4', type: 'tool', name: 'write-output', status: 'failed',
      metadata: { error: 'Disk full' } },
  ],
  edges: [
    { from: 'n1', to: 'n2', type: 'sequence' },
    { from: 'n2', to: 'n3', type: 'branched',
      metadata: { alternatives: ['inline-validate', 'skip-validation'] } },
    { from: 'n3', to: 'n4', type: 'retried',
      metadata: { retry_count: 2, final_outcome: 'failed' } },
  ],
  trace_events: [],
};

const decisions = extractDecisionsFromGraph(graph);

// Returns 5 decisions:
// [
//   { type: 'tool_choice', choice: 'fetch-data', outcome: 'completed',
//     node_id: 'n1', agent_id: 'invoice-agent' },
//   { type: 'tool_choice', choice: 'transform-payload', outcome: 'completed',
//     node_id: 'n2', agent_id: 'invoice-agent' },
//   { type: 'branch', choice: 'validator-bot',
//     alternatives: ['inline-validate', 'skip-validation'],
//     node_id: 'n3', agent_id: 'invoice-agent' },
//   { type: 'delegation', choice: 'validator-bot', parent_agent: 'invoice-agent',
//     node_id: 'n3', agent_id: 'invoice-agent' },
//   { type: 'retry', choice: 'write-output', retry_count: 2,
//     final_outcome: 'failed', node_id: 'n4', agent_id: 'invoice-agent' },
//   { type: 'failure', choice: 'write-output', error: 'Disk full',
//     failure_path: ['n1', 'n2', 'n3', 'n4'], node_id: 'n4', agent_id: 'invoice-agent' }
// ]

Converting decisions to vault entities

import { decisionsToEntities } from './decision-extractor';

const entities = decisionsToEntities(decisions, graph.id);

// Each decision becomes an L1 entity:
// {
//   type: 'decision',
//   id: 'tool-choice-fetch-data-invoice-agent',     // Stable ID from graph+node
//   name: 'tool_choice: fetch-data (invoice-agent)',
//   status: 'active',
//   layer: 'archive',
//   source_worker: 'harvester',
//   decision_type: 'tool_choice',
//   choice: 'fetch-data',
//   outcome: 'completed',
//   agent_id: 'invoice-agent',
//   graph_id: 'exec-invoice-agent-20260321-001',
//   trace_id: 'decision-exec-invoice-agent-20260321-001-n1',
//   confidence: 'medium',
//   tags: ['graph-inferred', 'tool_choice'],
// }

Harvester graph ingestion

import { Harvester } from './harvester';

const harvester = new Harvester(vault, config);

// ingestGraph handles the full pipeline:
// 1. Extract decisions from graph structure
// 2. Convert to entities with stable IDs
// 3. Deduplicate against existing vault entries (by trace_id)
// 4. Write new decision entities to L1
const result = await harvester.ingestGraph(graph);

// result: { created: 4, skipped: 2, total: 6 }
// Skipped entries already existed (idempotent on re-run)

Optional explicit decision events

// Frameworks can optionally emit richer decision data
const graphWithExplicit = {
  ...graph,
  trace_events: [
    {
      type: 'decision',
      timestamp: '2026-03-21T01:00:00.000Z',
      data: {
        choice: 'validator-bot',
        alternatives: ['inline-validate', 'skip-validation'],
        rationale: 'External validator has higher accuracy for financial data',
        confidence: 0.9,
      } as DecisionTraceData,
    },
  ],
};

// Explicit events supplement graph-inferred decisions
// They get higher confidence and include rationale

Cross-agent decision synthesis

import { Synthesizer } from './synthesizer';

const synthesizer = new Synthesizer(vault, config);

// Groups decisions by type+choice across agents
// Detects behavioral patterns: "All agents prefer tool X"
const patterns = await synthesizer.synthesizeDecisions();

// Creates L3 proposals like:
// {
//   type: 'synthesis',
//   name: 'Cross-agent pattern: fetch-data tool preference',
//   confidence: 0.85,    // 6 agents corroborate
//   evidence_links: ['decision-tc-fetch-invoice-agent', 'decision-tc-fetch-email-router', ...],
//   layer: 'emerging'
// }

Rules and Invariants

Five decision types

Graph Structure	Decision Type	Fields Captured
tool node	tool_choice	Tool name, metadata, duration, outcome
branched edge	branch	Selected branch, sibling alternatives
retried edge	retry	Retry count, final outcome
subagent node	delegation	Subagent name, parent agent
Failed node	failure	Error message, failure path from root

Optional explicit events

Frameworks can emit decision trace events with DecisionTraceData for richer data:

interface DecisionTraceData {
  choice: string;
  alternatives?: string[];
  rationale?: string;
  confidence?: number;
}

Explicit events supplement graph-inferred decisions — they do not replace them. Both sources are written to L1.

Framework-agnostic guarantee

Decision extraction works from ExecutionGraph structure (nodes, edges, metadata), not from adapter-specific events. The contract is:

• Any framework that produces an ExecutionGraph gets decision extraction for free.
• No adapter changes required.
• No configuration required.
• The Harvester walks the graph and infers decisions from topology.

Deduplication and idempotency

• Decision entity IDs are stable: derived from graph_id + node_id.
• Re-ingesting the same graph produces the same IDs, so duplicates are detected and skipped.
• The Harvester checks trace_id against existing vault entries before creating.

Confidence scoring for decisions

Source	Confidence
Graph-inferred (single agent)	medium
Explicit trace event	Uses provided confidence or high
Cross-agent synthesis (5+ agents)	>= 0.8
Single-agent synthesis	Capped at 0.5

Circuit breakers

• The Harvester stops after 100 entity creates per ingestGraph() call.
• The Synthesizer stops after 100 L3 proposals per synthesizeDecisions() call.
• These limits prevent runaway creation from pathologically large graphs.