The Model Is Not the Product: Residual State, Compiled Agents, and the Optimization of Loops

July 3, 2026

---

The model is no longer the product. The loop is.

That idea keeps getting reinforced every time I look at new research from Apple, LMSYS, and the recent work on autoresearch and constrained optimization. They're not converging on a better chatbot. They're converging on something closer to a reconfigurable system of computation where "reasoning" is just one phase inside a larger machine.

What's changing isn't just capability. It's where intelligence lives.

It's shifting out of the model and into three places at once: residual state, execution graphs, and optimization loops.

This isn't abstract. Each of these threads has concrete implementations — and when you wire them together, you get something that looks less like a chatbot and more like a continuously recompiled cognitive engine. I've been building toward this architecture across several systems: Objective05 (persistent intelligence infrastructure in Rust), Sovereign Memory Bank (7-layer autonomous cognitive memory), Dynamic Persona MoE RAG (persona-driven mixture-of-experts over local graphs), and SovereignSpec (spec-driven development with GraphRAG). This post is the synthesis of what those systems are converging on — and what the research confirms.

---

1. From Tokens to Residual State

Apple's Residual Context Diffusion

Apple's Residual Context Diffusion (RCD) quietly breaks one of the core assumptions behind most LLM systems: that intermediate uncertainty should be discarded.

Paper: Residual Context Diffusion Language Models — arXiv:2601.22954 (Hu et al., 2026) Code: github.com/yuezhouhu/residual-context-diffusion

In standard generation pipelines, we sample, reject, and move on. Low-confidence paths disappear. Only the final sequence matters.

RCD changes that. Instead of throwing away "failed" intermediate states during diffusion, it feeds them forward as contextual residuals — entropy-weighted continuous embedding vectors injected into subsequent denoising steps.

Here's the core mechanism in pseudocode:

pythonCopy
1import torch
2import torch.nn.functional as F
3
4def residual_diffusion_step(
5    x_t: torch.Tensor,           # masked embedding at step t
6    logits: torch.Tensor,        # model logits over vocabulary
7    embed_weight: torch.Tensor,  # vocabulary embedding matrix
8    residual_buffer: list,       # accumulated residuals from prior steps
9    temperature: float = 1.0,
10    entropy_threshold: float = 0.5,
11) -> tuple[torch.Tensor, torch.Tensor]:
12    """
13    One step of RCD decoding.
14
15    Instead of hard-committing to argmax tokens and discarding the rest,
16    RCD converts the full predictive distribution into a residual vector
17    and feeds it forward into the next step.
18    """
19    # Compute token probabilities
20    probs = F.softmax(logits / temperature, dim=-1)
21
22    # Entropy-weighted residual: sum over vocab weighted by uncertainty
23    # High-entropy (uncertain) positions contribute more residual signal
24    entropy = -(probs * torch.log(probs + 1e-8)).sum(dim=-1, keepdim=True)
25    normalized_entropy = entropy / entropy.max()
26
27    # Residual = weighted sum of all vocabulary embeddings
28    # NOT just the argmax token — every candidate contributes
29    residual = torch.einsum("b v, v d -> b d", probs, embed_weight)
30    residual = residual * (normalized_entropy > entropy_threshold).float()
31
32    # Accumulate residual into buffer
33    residual_buffer.append(residual.detach())
34
35    # Blend: combine original masked embedding with residual history
36    # The mixing weight is itself entropy-dependent
37    blend_weight = torch.sigmoid(2.0 * normalized_entropy - 1.0)
38    x_next = (1 - blend_weight) * x_t + blend_weight * residuals.mean(dim=0)
39
40    return x_next, probs

That sounds like a small tweak. It isn't.

Results: RCD achieves 5–10 point accuracy gains on frontier diffusion LLMs, nearly 2× baseline on AIME, and 4–5× fewer denoising steps at equivalent accuracy — all from converting a standard dLLM with ~300M tokens of additional training. The paper shows this works because the residual buffer captures discarded hypotheses, low-probability reasoning paths, and partial structures that didn't resolve cleanly — everything we normally optimize away becomes state for the next iteration.

What This Means for System Architecture

In most LLM systems (including RAG), memory is treated as retrieval:

pythonCopy
1def standard_rag(query: str, top_k: int = 5) -> str:
2    embedding = embedder.embed(query)
3    results = vector_store.similarity_search(embedding, k=top_k)
4    return format_context(results)

But RCD suggests a different model:

Memory is not retrieval. Memory is residue.

In my Sovereign Memory Bank architecture (post, repo), I implemented exactly this principle through the 7-layer memory hierarchy. Layer 0 (source) and Layer 1 (extracted concepts/claims/entities) are the residual accumulation layer — nothing is discarded, everything feeds forward:

pythonCopy
1# From Sovereign Memory Bank's memory hierarchy:
2# Every extraction round preserves all intermediate representations
3# as first-class graph nodes, regardless of "confidence"
4
5class ExtractedClaim(BaseModel):
6    text: str
7    source_chunk_id: str
8    confidence: float  # low-confidence claims are NOT filtered — they persist
9    residual_embedding: list[float]  # distributional residual, not just argmax
10    extraction_round: int  # provenance for evolution tracking
11    status: Literal["candidate", "verified", "contradicted", "superseded"]

The principle is structural: even failure becomes state. In the context of Dynamic Persona MoE RAG (post, repo), this means a persona that produces a low-confidence response doesn't get ignored — its partial output feeds into the next persona's conditioning. The activation_cost and historical_performance fields on each persona schema become the residual signal that shapes future routing decisions.

---

2. From Tool Use to Executable Systems

LMSYS and SGLang Agents

The LMSYS work on agent-assisted SGLang development pushes the next abstraction shift: the agent is no longer just a consumer of tools. It becomes part of the system that defines execution.

Paper: SGLang: Efficient Execution of Structured Language Model Programs — arXiv:2312.07104 (Zheng et al., NeurIPS 2024) Repo: github.com/sgl-project/sglang (29.9k+ stars, 400k+ GPUs in production)

Instead of:

pythonCopy
1prompt → model → tool call → result

We start seeing:

pythonCopy
1agent → compiles execution graph → optimizes inference paths → rewrites runtime behavior → executes

SGLang already treats inference as a structured program through its Python-embedded DSL with primitives like gen, select, fork, join, and extend. What the agent layer adds is adaptability at the level of the execution graph itself.

Here's how SGLang represents a multi-step inference as a compilable graph:

pythonCopy
1import sglang as sgl
2
3@sgl.function
4def multi_step_reasoning(context: str, question: str):
5    """
6    SGLang compiles this into a computational graph
7    that the runtime can optimize via code motion,
8    instruction selection, and auto-tuning.
9    """
10    # Step 1: Analyze context
11    analysis = sgl.gen("analysis", max_tokens=256)
12    
13    # Step 2: Fork — explore multiple reasoning paths in parallel
14    fork_context = sgl.fork(3)
15    with fork_context:
16        hypothesis_1 = sgl.gen("path_1", max_tokens=128, temperature=0.3)
17        hypothesis_2 = sgl.gen("path_2", max_tokens=128, temperature=0.7)
18        hypothesis_3 = sgl.gen("path_3", max_tokens=128, temperature=0.9)
19    
20    # Step 3: Join — synthesize across paths
21    sgl.join()
22    synthesis = sgl.gen("synthesis", max_tokens=256)
23    
24    # Step 4: Constrained decode — output must match JSON schema
25    final = sgl.gen(
26        "final",
27        max_tokens=512,
28        schema={
29            "type": "object",
30            "properties": {
31                "answer": {"type": "string"},
32                "confidence": {"type": "number"},
33                "reasoning_paths": {
34                    "type": "array",
35                    "items": {"type": "string"}
36                }
37            },
38            "required": ["answer", "confidence"]
39        }
40    )
41    return final

The runtime applies RadixAttention — a radix-tree LRU cache for KV tensors that enables automatic prefix reuse across calls. If the same context prefix appears in a later query, the KV cache is reused rather than recomputed. This gives up to 6.4× higher throughput vs vLLM on agent/reasoning/RAG/multi-turn workloads.

The critical architectural property: the execution graph is mutable at runtime. An agent can observe its own inference pattern and rewrite the execution graph — adding branches, merging paths, reordering operations — by generating new SGLang programs that describe the next iteration's structure.

What This Means for System Architecture

This matters because it dissolves the boundary between "model reasoning" and "system architecture."

The agent is no longer sitting on top of the stack. It is partially responsible for constructing the stack on each run.

Most agent frameworks today assume:

• Static tool definitions
• Fixed orchestration logic
• Stable execution pipelines

But real systems under SGLang-style design become:

• Dynamic execution graphs
• Query-dependent compilation
• Runtime-optimized inference paths

In my Objective05 architecture (post, repo), this maps directly onto the EventEngine and SchedulerService pattern. The scheduler emits typed events onto a message bus; pipeline workers consume events and dispatch to ingestion/extraction/correlation/maintenance functions. The critical insight from Objective05 is that the pipeline topology itself is query-dependent — different document types trigger different extraction chains, and the correlation engine's merge logic is parameterized by entity type and temporal proximity:

rustCopy
1// From Objective05's EventEngine design:
2// Execution paths are not fixed — they're compiled per event type
3
4enum PipelineStage {
5    Ingest { source: SourceAdapter },
6    Extract { method: ExtractionMethod },   // heuristic vs LLM
7    Correlate { threshold: f64 },           // merge threshold per entity type
8    Maintain { action: MaintenanceAction }, // archive, promote, notify
9}
10
11struct ExecutionGraph {
12    stages: Vec<PipelineStage>,             // compiled per event class
13    cache_hint: Option<RadixKey>,           // KV cache strategy
14    timeout: Duration,                      // budget constraint
15}
16
17impl SchedulerService {
18    fn compile_graph(&self, event: &Event) -> ExecutionGraph {
19        // The execution graph is generated — not hardcoded
20        match event.class {
21            EventClass::Financial => ExecutionGraph {
22                stages: vec![
23                    PipelineStage::Extract { method: ExtractionMethod::LLM },
24                    PipelineStage::Correlate { threshold: 0.85 },
25                ],
26                cache_hint: Some(RadixKey::from(event.entity_id())),
27                timeout: Duration::from_secs(30),
28            },
29            EventClass::Social => ExecutionGraph {
30                stages: vec![
31                    PipelineStage::Extract { method: ExtractionMethod::Heuristic },
32                    PipelineStage::Correlate { threshold: 0.6 },
33                    PipelineStage::Maintain { action: MaintenanceAction::Flag },
34                ],
35                cache_hint: None,  // social events have low cache reuse
36                timeout: Duration::from_secs(10),
37            },
38        }
39    }
40}

Which leads to a harder conclusion:

If the execution graph is mutable, then "the system" is not static software. It is a generated artifact. And agents are compilers.

---

3. From Loops to Constrained Optimization

Autoresearch Systems as Formal Search Spaces

The third piece is more subtle, but it completes the picture.

Autoresearch-style systems framed through constrained optimization treat agent loops not as "iteration until better answer," but as structured search over a bounded space of possible reasoning trajectories.

Key references:

• Karpathy's autoresearch — github.com/karpathy/autoresearch: minimal agent loop (700 experiments in 2 days on H100, 20 optimizations discovered)
• Bilevel Autoresearch (Qu & Lu, 2026) — arXiv:2603.23420: outer loop optimizes inner loop's search mechanism
• Agent Contracts (Ye & Tan, 2026) — arXiv:2601.08815: formal resource-bounded agent execution with conservation laws
• CCPO (Si et al., 2026) — arXiv:2511.11828: conformal constrained policy optimization for cost-effective agents
• EvoTrainer (2026) — arXiv:2606.03108: co-evolving LLM policies and training harnesses

Instead of:

pythonCopy
1generate → critique → refine → repeat

We get:

pythonCopy
1explore hypothesis space → evaluate against constraints → allocate compute dynamically → converge under budgeted uncertainty

Here's what that looks like as a formal optimization loop:

pythonCopy
1from dataclasses import dataclass
2from enum import Enum
3from typing import Callable, Generic, TypeVar
4
5State = TypeVar("State")
6Action = TypeVar("Action")
7
8class ResourceConstraint(Enum):
9    TOKENS = "tokens"
10    API_CALLS = "api_calls"
11    ITERATIONS = "iterations"
12    LATENCY_MS = "latency_ms"
13    COST_USD = "cost_usd"
14
15@dataclass
16class Budget:
17    """Formal resource budget from the Agent Contracts framework."""
18    limits: dict[ResourceConstraint, float]
19    consumed: dict[ResourceConstraint, float]
20
21    def remaining(self, constraint: ResourceConstraint) -> float:
22        return self.limits.get(constraint, float("inf")) - self.consumed.get(constraint, 0.0)
23
24    def within_budget(self) -> bool:
25        return all(
26            self.consumed.get(k, 0.0) <= v
27            for k, v in self.limits.items()
28        )
29
30@dataclass
31class Trajectory:
32    """A complete reasoning trajectory, not just the final answer."""
33    steps: list[tuple[State, Action, float]]  # state, action, reward
34    total_cost: float
35    total_tokens: int
36
37class ConstrainedOptimizationLoop(Generic[State, Action]):
38    """
39    An agent loop framed as constrained optimization over
40    a bounded space of reasoning trajectories.
41
42    Reference: CCPO (Si et al., 2026) + Agent Contracts (Ye & Tan, 2026)
43    """
44
45    def __init__(
46        self,
47        explore_policy: Callable[[State, Budget], Action],
48        exploit_policy: Callable[[State, Budget], Action],
49        evaluate: Callable[[Trajectory], float],
50        budget: Budget,
51        confidence_target: float = 0.95,
52    ):
53        self.explore = explore_policy   # cheap model for exploration
54        self.exploit = exploit_policy   # expensive model for exploitation
55        self.evaluate = evaluate        # objective function
56        self.budget = budget
57        self.confidence_target = confidence_target
58        self.trajectories: list[Trajectory] = []
59
60    def step(self, state: State) -> Action:
61        """Adaptive action selection based on remaining budget and uncertainty."""
62        uncertainty = self._estimate_uncertainty(state)
63        remaining = self.budget.remaining(ResourceConstraint.TOKENS)
64
65        # The explore/exploit decision is itself computed — not hardcoded
66        if uncertainty > self.confidence_target and remaining > 1000:
67            # Explore: cheap model, wide search
68            return self.explore(state, self.budget)
69        else:
70            # Exploit: expensive model, precise answer
71            return self.exploit(state, self.budget)
72
73    def _estimate_uncertainty(self, state: State) -> float:
74        """Conformal prediction over past trajectory outcomes."""
75        if len(self.trajectories) < 10:
76            return 1.0  # maximum uncertainty, always explore early
77        return 1.0 - self._coverage_estimate()
78
79    def _coverage_estimate(self) -> float:
80        """Empirical coverage of correct answers in prediction sets."""
81        correct = sum(1 for t in self.trajectories[-20:] if t.steps[-1][2] > 0.5)
82        return correct / min(len(self.trajectories), 20)

The key shift is that the loop is no longer informal.

It has geometry:

• Constraints (compute, latency, hallucination risk, API budget)
• Objectives (accuracy, novelty, coherence, utility)
• Tradeoffs between exploration and exploitation, formalized via conformal prediction

This is important because it forces something most agent systems avoid: you have to define what "better" actually means in system terms.

In my Building Autonomous Sovereign AI post, I described this as the Inner Loop vs Outer Loop separation. The Inner Loop executes tasks; the Outer Loop observes performance and drives improvement. The Karpathy-style autoresearch system is the purest form — a model edits its own training script, runs for exactly 5 minutes, measures val_bpb, and keeps or discards the change. The binary keep/discard criterion is the objective function. The 5-minute wall-clock window is the resource constraint. The single editable file (train.py) is the trust boundary.

pythonCopy
1# From Karpathy's autoresearch pattern:
2# The loop is the product. The model is a component.
3
4class AutoresearchLoop:
5    """
6    Minimal constrained optimization over research trajectories.
7
8    Key design decisions:
9    - Fixed time budget per experiment (5 min)
10    - Single mutable file (train.py)
11    - Binary acceptance criterion (val_bpb improvement)
12    """
13
14    def __init__(self, experiment_dir: Path, time_budget_s: int = 300):
15        self.experiment_dir = experiment_dir
16        self.time_budget_s = time_budget_s
17        self.history: list[ExperimentResult] = []
18        self.best_bpb = float("inf")
19
20    def propose_change(self, model, context: str) -> str:
21        """Generate a code change hypothesis."""
22        prompt = f"""Current validation bits-per-byte: {self.best_bpb:.4f}
23History: {len(self.history)} experiments, {'improving' if self._trend() > 0 else 'plateauing'}
24
25Propose a single-file change to train.py that could improve val_bpb.
26Strategy: {context}
27Return ONLY the diff."""
28        diff = model.generate(prompt)
29        return diff
30
31    def execute(self, diff: str) -> ExperimentResult:
32        """Apply change, run with fixed budget, measure outcome."""
33        backup = (self.experiment_dir / "train.py").read_text()
34        try:
35            # Apply proposed change
36            result = subprocess.run(
37                ["git", "apply"],
38                input=diff, text=True, capture_output=True, cwd=self.experiment_dir
39            )
40            if result.returncode != 0:
41                return ExperimentResult(diff=diff, val_bpb=None, accepted=False)
42
43            # Run with hard time budget
44            start = time.time()
45            proc = subprocess.run(
46                ["python", "train.py"],
47                timeout=self.time_budget_s, cwd=self.experiment_dir,
48                capture_output=True, text=True
49            )
50            elapsed = time.time() - start
51
52            # Parse validation metric
53            val_bpb = self._parse_val_bpb(proc.stdout)
54            accepted = val_bpb is not None and val_bpb < self.best_bpb
55            if accepted:
56                self.best_bpb = val_bpb
57
58            return ExperimentResult(
59                diff=diff, val_bpb=val_bpb,
60                elapsed_s=elapsed, accepted=accepted
61            )
62        finally:
63            (self.experiment_dir / "train.py").write_text(backup)
64
65    def _trend(self) -> float:
66        if len(self.history) < 5:
67            return 0.0
68        recent = [r.val_bpb for r in self.history[-5:] if r.val_bpb is not None]
69        return (recent[0] - recent[-1]) / len(recent) if len(recent) >= 2 else 0.0

The Bilevel Autoresearch extension (Qu & Lu, 2026) takes this further: the outer loop doesn't just propose changes to the training script — it generates the search strategy itself (Tabu Search, Bandit, Orthogonal Exploration) as executable Python code, achieving 5× improvement over the inner loop alone.

---

4. Putting It Together: Residual Systems, Compiled Agents, and Optimization Loops

These three threads — Apple's residual diffusion framing, LMSYS's execution graph agents, and autoresearch/constrained optimization — aren't separate ideas. They're three layers of the same transition.

They map cleanly onto a new system stack that my projects are converging on:

Layer 1: Residual State (Memory)

• Not retrieval-based RAG
• But persistent accumulation of: failed reasoning, partial structures, unresolved hypotheses
• Memory becomes a living substrate, not a lookup table

In Sovereign Memory Bank (post, repo), this is Layers 0–2 of the 7-layer hierarchy: source documents → extracted concepts/claims/entities → structured relationships. The evolution engine autonomously merges, splits, promotes, and deprecates nodes based on incoming evidence — exactly the RCD principle applied at system scale.

Layer 2: Execution Graphs (Compute)

• Not static tool calling
• But dynamic inference compilation
• Agents participate in shaping runtime structure
• Execution becomes query-specific and mutable

In SovereignSpec (post, repo), this is the 12-step compilation pipeline: parse → validate → resolve_deps → check_contradictions → compute_drift → generate_plan → generate_tasks → generate_context → generate_docs → update_knowledge_graph → update_embeddings → commit_version. Each invocation compiles a new execution graph for the spec being processed, with the GraphEngine computing dependency chains and impact analysis via NetworkX.

In Objective05 (post, repo), the SchedulerService emits typed events onto a message bus — pipeline workers consume and dispatch to ingestion/extraction/correlation/maintenance functions. The pipeline topology per event is not hardcoded; it's compiled per event class with different cache strategies, timeouts, and processing chains.

Layer 3: Constrained Loops (Reasoning)

• Not open-ended generation
• But optimization over trajectories
• Bounded by compute, uncertainty, and objective functions

In the Dynamic Persona MoE RAG system (post, repo), the routing decision (which persona to activate for a query) is itself a constrained optimization. Each persona has an activation_cost; the router balances persona expertise against total compute budget. The historical_performance field feeds back into the routing policy, creating the explore/exploit dynamic formalized by CCPO.

pythonCopy
1# From Dynamic Persona MoE RAG: persona routing as constrained optimization
2
3@dataclass
4class Persona:
5    name: str
6    expertise: list[str]
7    traits: dict[str, float]  # 1-9 scale
8    activation_cost: int      # tokens consumed per invocation
9    historical_performance: float  # running accuracy score
10    last_activated: float      # timestamp for recency weighting
11
12class ConstrainedPersonaRouter:
13    """
14    Routes queries to personas under resource constraints.
15
16    This is the Layer 3 (Constrained Loops) instantiation in MoE RAG.
17    """
18
19    def __init__(self, personas: list[Persona], budget: Budget):
20        self.personas = personas
21        self.budget = budget
22
23    def select(
24        self,
25        query: str,
26        query_embedding: list[float],
27        top_k: int = 3,
28    ) -> list[Persona]:
29        """
30        Select top-k personas under budget constraints.
31
32        Uses a scoring function that blends:
33        1. Semantic similarity (query → persona expertise)
34        2. Activation cost penalty
35        3. Historical performance bonus
36        4. Recency bonus (favor recently validated personas)
37        """
38        candidates = []
39        remaining_tokens = self.budget.remaining(ResourceConstraint.TOKENS)
40
41        for persona in self.personas:
42            if persona.activation_cost > remaining_tokens:
43                continue  # prune — violates budget constraint
44
45            # Similarity + cost-aware scoring
46            expertise_sim = cosine_similarity(query_embedding, persona_expertise_embedding(persona))
47            cost_penalty = persona.activation_cost / 1000
48            perf_bonus = persona.historical_performance * 0.3
49
50            score = expertise_sim - cost_penalty + perf_bonus
51            candidates.append((score, persona))
52
53        candidates.sort(key=lambda x: x[0], reverse=True)
54        selected = [p for _, p in candidates[:top_k]]
55
56        # Deduct from budget
57        total_cost = sum(p.activation_cost for p in selected)
58        self.budget.consumed[ResourceConstraint.TOKENS] = (
59            self.budget.consumed.get(ResourceConstraint.TOKENS, 0) + total_cost
60        )
61
62        return selected

The Unified Architecture

When combined, something new emerges:

A system where intelligence is not located in the model, but in the interaction between:

• residual memory (Sovereign Memory Bank, RCD)
• compiled execution (SGLang agents, Objective05 EventEngine, SovereignSpec pipeline)
• constrained iteration (autoresearch loops, CCPO, Dynamic MoE RAG router)

This is the architecture I described in the Sovereign Synthesis — the 7-layer unified architecture where Interface (Next.js) → API (FastAPI) → Orchestration (MoE/DeerFlow) → Governance (Control Boundary) → Reasoning (Persona Engine/SpecGen) → Memory (NetworkX + ChromaDB) → Inference (Ollama/llama.cpp). The three layers in this post (residual state, execution graphs, constrained loops) map directly onto the Memory, Reasoning, and Governance layers of the Synthesis.

---

5. What This Implies for Agent Systems

If you're building something like Dynamic Persona MoE RAG, Hermes-style orchestration, Objective05, Sovereign Memory Bank, or any autoresearch loop system, the implication is simple but uncomfortable:

Most current architectures are only simulating parts of this stack.

• RAG simulates memory, but discards residue
• Tool-using agents simulate execution, but don't compile graphs
• Prompt loops simulate optimization, but don't formalize objectives

The next step is not "better prompting" or "bigger models."

It is structural:

1. Stop treating failed outputs as waste. Treat them as state. — Implement residual accumulation (RCD-style) in your memory layer. Sovereign Memory Bank's evolution engine does this; so does Objective05's event merge pattern.

2. Stop treating execution as fixed. Treat it as compiled per query. — Generate execution graphs at runtime. Objective05's compile_graph() and SovereignSpec's 12-step pipeline are examples. SGLang shows how to make this efficient with RadixAttention.

3. Stop treating iteration as narrative. Treat it as constrained optimization. — Formalize your budget, objective function, and explore/exploit policy. CCPO and Agent Contracts provide the mathematical framework. Karpathy's autoresearch shows the minimal viable implementation.

Once you do that, the system stops looking like an LLM application.

It starts looking like a continuously recompiled cognitive engine.

---

6. Closing

The model is not the product anymore because it was never the full system in the first place.

What's emerging now is something closer to a programmable cognition substrate:

• memory that accumulates error as structure
• execution that compiles itself per task
• reasoning that optimizes under constraint

In that world, the interesting question is no longer:

"What can the model do?"

It becomes:

"What kind of loop are you running, and what does it optimize for?"

And that's where everything starts to converge.

---

References

1. Hu et al. Residual Context Diffusion Language Models. arXiv:2601.22954, 2026
2. Zheng et al. SGLang: Efficient Execution of Structured Language Model Programs. arXiv:2312.07104, NeurIPS 2024
3. Karpathy. autoresearch. GitHub, 2026
4. Qu & Lu. Bilevel Autoresearch. arXiv:2603.23420, 2026
5. Ye & Tan. Agent Contracts: A Formal Framework for Resource-Bounded Autonomous AI Systems. arXiv:2601.08815, 2026
6. Si et al. Conformal Constrained Policy Optimization for Cost-Effective LLM Agents. arXiv:2511.11828, AAAI 2026
7. Harris & Slivkins. Should You Use Your LLM to Explore or Exploit?. arXiv:2502.00225, 2026
8. EvoTrainer. Co-Evolving LLM Policies and Training Harnesses. arXiv:2606.03108, 2026

Related Posts

• The Model Is Not the Product: On Building Persistent Intelligence Infrastructure — Objective05 deep dive
• Building Autonomous Sovereign AI: Autoresearch Loops and Expert Fine-Tuning — Inner/outer loop architecture
• Sovereign Memory Bank: Autonomous Cognitive Memory for Agent Systems — 7-layer memory hierarchy
• Dynamic Persona MoE RAG: Building Memory-Driven Synthetic Intelligence — Persona-based constrained routing
• SovereignSpec: Local-First Spec-Driven Development — 12-step compilation pipeline
• SOVEREIGN: The Unified Architecture — 7-layer convergent architecture
• Context Engineering: The Blind Spots and the Real Full-Stack Development Paradigm — Agent harnesses and persistent memory

Related Posts

• The Sovereign Intelligence Stack — Architecture implementation with working code
• The Loop Is the Product — Intelligence Observatory deep dive
• Building Autonomous Sovereign AI — Autoresearch loops and expert fine-tuning
• Getting Started with Sovereign AI — Beginner on-ramp
• Local AI Architecture — Local-first implementation guide
• Retrieval Architecture — Memory and retrieval systems

Related Repositories

• sovereign-intelligence-stack — 70 Python files, 7,757 lines
• sovereign-memory-bank — Autonomous cognitive memory
• dynamic-persona-moe-rag — Persona-driven MoE
• objective05 — Rust persistent intelligence infrastructure
• sovereignspec — Spec-driven development

Additional Research

• LMSYS/SGLang — Agentic execution graphs
• Bridgewater AIA Labs — Autonomous evaluation
• Thinking Machines Lab — Compounding intelligence
• Apple Research — Residual Context Diffusion team
• Autonomous Agent Research — arXiv search
• Compounding Intelligence Research — arXiv search