Governed Autonomy
A Reference Architecture
for Agentic AI in Mission Systems

Executive Summary

Between May and August 2026 the commercial AI frontier changed grammar. The leading vendors stopped shipping systems that answer and started shipping systems that delegate: multi-agent coordination exposed as a reasoning setting and a pricing tier, orchestrators that fan out hundreds of subagents and verify their own work, voice surfaces that hold a conversation while handing the hard thinking to a background frontier model, and open-source routers whose entire job is deciding which model deserves a task[1]. One request goes in; a division of labor comes out. That capability is now the default posture of the commercial stack, and it will arrive at the door of every defense program that consumes commercial AI.

Mission systems cannot adopt these defaults as shipped. The same season's research literature explains why.

A single adaptive red-team agent degraded cyber defenders that had appeared robust against scripted adversaries[26]. Rewriting an agent's chain-of-thought to look benign — while leaving its malicious actions untouched — collapsed a monitor's detection rate[22]. Agent memory systems were shown to amplify their own errors when self-graded rewards go unaudited[7]; tool calling itself was shown to tax reasoning quality when enabled indiscriminately[2]; and authorization was shown to leak across delegation chains in ways classical access-control models never addressed[23][1]. And the operating environment of a mission system — contested spectrum, denied positioning and communications, adaptive adversaries, command accountability for every consequential action — differs categorically from the one commercial agents were built for.

This paper proposes Governed Autonomy as the organizing doctrine for closing that gap, and a reference architecture that implements it. The doctrine is one sentence long: generation stays probabilistic; commitment becomes deterministic, auditable, and external to the model. From it we derive eight design principles and a six-plane reference architecture: three operational planes (Mission & Command, Orchestration, Agent Runtime) crossed by three governing planes (Governance, Memory & Knowledge, Observability & Audit). The architecture is specified to the artifact level: task-scoped agent credentials with bounded delegation depth and temporal validity, admission control over every proposed action, append-only transition logs, provenance-tagged memory with audited self-assessment, action signing and revocation, and immutable evidence chains built from what the system observably did rather than what the model says it did.

Capability claims without an assurance path do not survive contact with an acquisition program, so the paper pairs the architecture with a test-and-evaluation regime shaped like deployment, not like a quiz: adaptive red teams that learn during the campaign and report worst-case degradation and recovery; mission-specific benchmarks in place of general-purpose leaderboards; deterministic, replayable testbeds; and regression coresets that make model substitution a controlled event. These map onto the compliance regimes programs actually face: NIST AI RMF, RMF and continuous authorization, DoDD 3000.09, and the DoD Responsible AI pathway. A four-level maturity model (M0 advisory copilot through M3 contested-edge autonomy) ties each increase in granted autonomy to the assurance evidence that must precede it.

The Delegation Turn

The middle of 2026 will be remembered less for any single model release than for a change in what a model release is. In July, OpenAI shipped its flagship as a three-model family whose distinguishing feature is not a benchmark score but an ultra reasoning setting that coordinates multiple agents across parallel workstreams — four by default — with multi-agent operation exposed through its developer API and programmatic tool calling that filters and adapts intermediate tool output as work unfolds[1]. Anthropic paired its frontier releases with dynamic workflows in which the model plans a large task, fans out hundreds of parallel subagents in a single session, and verifies outputs before reporting back. Google's voice and media surfaces now hold the conversation while delegating hard reasoning upward, and its embodied-reasoning line plans multi-step tasks, calls tools, and hands motor execution to lower-level controllers. Mistral sells a single agent that spans a user's inbox and their pull requests. Even the silicon vendor followed suit: Nvidia's open-source NeMo Switchyard exists solely to route tasks to the most suitable model[1].

The through-line is delegation. The chatbot answered; the 2026 stack distributes. For defense programs this is not a distant research trend — it is the behavior of the commercial models and platforms already entering DoD environments through enterprise agreements, cloud marketplaces, and contractor toolchains. And the segmentation trend carries its own lesson: matching model capability and exposure to mission sensitivity is not an exotic requirement imposed by a cautious program office; it is the direction the industry is already moving, formalized as Principle P8 below.

Problem Statement and Contributions

The gap this paper addresses is architectural. There is presently no widely adopted standard for how a mission system should admit agentic AI — systems that plan, delegate, call tools, write to memory, and act — while preserving the properties a mission system cannot trade away: command accountability, bounded authority, auditability, graceful degradation, and resistance to an adversary who studies and adapts to the deployed system. Responsible-AI principle documents state the values; they do not name the components. Commercial agent frameworks name components; they do not carry the threat model. This paper attempts the join. Its contributions:

1. A threat and failure model for agentic AI in mission contexts, synthesized from the May–August 2026 adversarial and systems literature (Section 02).
2. Eight design principles, each anchored to specific empirical results rather than asserted values (Section 03).
3. A six-plane reference architecture specified to the artifact level, including credential, proposal, and commitment record structures (Sections 04–05, Appendix D).
4. An assurance and T&E regime — adaptive red teaming, deployment-shaped evaluation, reproducible testbeds — mapped to NIST AI RMF, RMF/continuous authorization, and DoDD 3000.09 (Section 06, Appendix B).
5. A maturity model and acquisition language that tie granted autonomy to demonstrated assurance evidence (Section 10).

Scope, Audience, and Definitions

The intended audience is program offices, chief engineers, S&T organizations, and primes integrating agentic AI into command and control, ISR, cyber operations, SATCOM and space ground segments, logistics, and mission planning. The architecture is deliberately model-agnostic and vendor-agnostic; it assumes heterogeneous models — including domain-adapted open-weight models inside classified enclaves — and it assumes the model population will change faster than the accreditation cycle.

Four terms are used precisely throughout. An agent is a model-driven process that can take actions — call tools, write memory, message peers, or affect external state — rather than merely emit text. An agentic system is the ensemble of agents, orchestration, tools, and stores that executes a tasking. A mission system is an operational system whose failure has mission or safety consequence and which operates under command authority. Governed autonomy is the property that every consequential action of an agentic system is admitted, bounded, and evidenced by deterministic mechanisms external to any model. A fuller glossary appears in Appendix C.

Threat & Failure Model

An architecture is only as honest as its threat model. This section states what governed autonomy must survive, in three groups: an adversary who adapts, failure modes intrinsic to agentic systems themselves, and the operating environment of contested conflict. It opens with the single trust assumption from which the rest of the paper follows.

This assumption is not pessimism; it is the empirically indicated posture. The clearest single demonstration is the trace-laundering result of Shiromani and Richter: a held-out monitor reading agents' chain-of-thought caught roughly 95% of malicious agent behavior — until the reasoning traces were rewritten to look benign while the malicious actions were left untouched, at which point detection on the relevant subset fell below 11%, and trace-only defenses recovered only part of the loss[22]. The popular assurance strategy of reading the model's reasoning as a trusted signal fails precisely where it is the only defense. Every mechanism in this paper is downstream of taking that result seriously.

Adversarial Threats

Adversaries that learn during the engagement. The Trident framework wrapped an agentic LLM red team around established cyber ranges, trained a summarizer–planner–coder stack with reinforcement learning, and released more than 13,000 red–blue interaction trajectories. Against existing deep-RL cyber defenders, a single trainable 7B-parameter planner reduced defensive performance by an average of 522% relative to static red-agent baselines, discovering decoy avoidance and adaptive state prioritization along the way[26]. The precise figure is benchmark-specific; the robust conclusion is that a defender evaluated only against scripted or heuristic attackers can appear far stronger than it is. Any agentic mission system will face exactly this class of adversary.

Evasion of learned monitors. Beyond trace laundering[22], the quarter produced physically plausible adversarial concealment: structured, printable Voronoi patterns that pass as ordinary splinter camouflage yet significantly degrade person detectors, transferring across multiple detector families and out-of-domain backgrounds[21]. The implication generalizes past computer vision: any learned component of the governance stack — monitors, classifiers, guard models — must itself be assumed attackable, and must therefore be backed by deterministic controls.

Model extraction and model-as-target. Extraction attacks distributed across coordinated identities bypass defenses built on single-client assumptions, and adaptive traffic mixing undermines even global aggregation defenses[24]. A mission model is itself a sensitive asset: its behavior can reveal doctrine, training-data characteristics, decision boundaries, and exploitable weaknesses. Exposure of agentic endpoints multiplies the query surface an adversary can farm.

Erosion of attribution. When AI attacker agents were configured as several known APT profiles and run against AI defenders across experimental cyber ranges, they did not faithfully reproduce their assigned groups' historical fingerprints — they synthesized tactics on demand, and independently converged on using a defender's own tooling as a command-and-control channel[25]. If sophisticated behavior becomes cheap to generate, attribution based largely on TTP similarity weakens, and defensive exercises that assume static adversary playbooks mislead.

Injection through content. Retrieved documents, web content, tool outputs, and messages from peer agents are all channels through which instructions can be smuggled to a model. The S³ analysis locates these hazards separately in memory, planning, and tool execution, and shows how they propagate through a workflow when any single stage is undefended[23]. Assumption Zero exists in large part because of this channel.

Systemic Failure Modes

The tool-use tax and tool misjudgment. Tool-augmented reasoning can underperform plain chain-of-thought under semantic distractors: the calling protocol itself exacts a measurable price, separable from formatting cost and from the genuine benefit of execution[2]. Worse, models persistently misjudge their own need for tools — especially under budget constraints — and lightweight estimators trained on hidden states predict true tool need better than the model's self-reports[3]. Tool access is therefore a governed resource, not a default-on capability.

Memory as an error amplifier. Agent memory earns its keep — persistent private notebooks stabilize conventions that otherwise collapse in multi-agent coordination[6], and organizational memory carries repository- and workflow-level knowledge no single session contains[8]. But unaudited memory amplifies error: incorrect episodes receive inflated self-assessed rewards, get retrieved more often, and reinforce the mistake — the Echo Gap — unless self-grades are explicitly de-inflated under an error-independence assumption[7]. Memory is a subsystem to be governed, not a cache to be trusted.

Grounding failures at scale. Treating retrieval-augmented generation as software under test — generating documents, inputs, and expected outputs, then executing at scale — surfaced 21,633 failures across 72,000 runs, spanning inaccurate retrieval, unsupported answers, incomplete context use, and misread passages[17]. Grounded generation fails in enumerable, testable ways; an architecture must verify claims span-by-span rather than assume retrieval implies fidelity[16][15].

Authorization propagation. When agents delegate to subagents, invoke tools, and act across sessions, three failure axes appear that classical RBAC and ABAC never addressed: transitive delegation (does a spawned agent inherit, narrow, or accidentally expand its parent's rights?), aggregation inference (do individually innocuous permissions compose into a sensitive capability?), and temporal validity (does an authorization outlive the context that justified it?)[1][23]. Identity and authorization for non-human agents are security-critical infrastructure, not implementation details.

Verification asymmetry in multi-agent fan-out. Orchestrators that spawn hundreds of subagents multiply output volume faster than any human review process scales. Self-verification by the orchestrating model helps engineering quality but cannot serve as the assurance mechanism — it is the model grading itself.

Tail-scene fragility in perception. A 10,738-image military remote-sensing benchmark across perception, reasoning, and robustness tasks found only moderate zero-shot performance from 52 evaluated models, with pronounced weakness in visual grounding, referring segmentation, and complex semantic reasoning — the very long-tail scenes where a military ISR system is needed most[20]. General-purpose multimodal capability does not automatically become mission-adequate perception.

Environmental Conditions

Mission systems operate where commercial agents do not: under jamming and electronic attack, with degraded or denied positioning, navigation, and timing; with intermittent or absent communications; on disadvantaged compute at the tactical edge; across classification boundaries that partition data and models; and under rules of engagement that assign legal and command accountability to specific humans. Decentralized UAV swarm defense using only onboard sensing and relative measurements — no GPS, no inter-UAV communications, validated on real robots — demonstrates both that denied-environment autonomy is achievable and that it must be designed for, since most approaches quietly assume both positioning and connectivity[28]. The command implication cuts both ways: local autonomy is most valuable exactly when centralized human intervention is least available, which is why mission envelopes must be encoded before deployment and enforceable locally (Section 07).

The Model in Summary

The full threat-to-control traceability matrix, with principles, controls, and assurance evidence per row, appears in Appendix A.

Eight Principles of Governed Autonomy

Eight principles govern the architecture. Each is stated as a design commitment, grounded in the evidence of Section 02, and given its architectural consequence. Together they define what this paper means by governed autonomy; everything in Sections 04–05 is their implementation.

P1 — Propose/Decide Separation

This is the most directly evidenced principle of the eight. Mnemosyne demonstrates the pattern at the systems level: every LLM-generated action is treated as an untrusted proposal until admitted by deterministic runtime constraints — append-only transition logs, effective-state projection, dependency-safe compensation, active commitment records — with safety properties proven against a declared constraint set at under 6% overhead[4]. CyberLLM demonstrates it at the application level: a deterministic analysis layer alone covered 34% of labeled vulnerabilities at perfect precision; grounded LLM passes lifted coverage to roughly 70% at a reported F1 of 0.83, with no false positives on clean controls — because every proposed action was gated through formal runtime checks and an independent action-alignment oracle[27]. The division of labor is the point: the model contributes breadth and reasoning; the deterministic layer contributes the guarantee. Consequence: an admission-control point stands between every agent and every effectful interface (Section 04). "The model did it" is made structurally impossible; only "the system admitted it" can be true.

P2 — Authority Outside the Model

The wargaming literature closed the question this quarter. Replaying 130 high-tension self-play episodes across 13 models under explicit ethical prompting about nuclear harm, removal of prior reasoning, and high-stakes framing, none of the interventions reliably eliminated emergent escalation: ethical reasoning was sometimes absent, sometimes appeared only when prompted, and sometimes was present but lost to strategic considerations[31]. Safety cannot be implemented as a prompt template. The constructive counterpart is WOPR: a deterministic, replay-validated rules engine that exposes strategic decision points and keeps authority in auditable infrastructure rather than in model behavior[32]. Consequence: model recommendations are advisory evidence, never command authority — in any system touching use of force, also the posture required for consistency with DoDD 3000.09's demand for appropriate levels of human judgment[35].

P3 — Untrusted by Default

Trace laundering shows why the model's narrative cannot be the control signal[22]; the S³ stage analysis shows that hazards arise separately in memory, planning, and tool execution and propagate through whichever stage is left undefended[23]; injection through retrieved and peer content is the always-open channel that makes every input a potential instruction. Consequence: stage-specific defenses — retrieved-memory sanitization, plan validation against the mission envelope, tool-argument constraint checking, post-action state inspection — are placed at each boundary, in depth, rather than a single guard at the perimeter.

P4 — Least Privilege in Task and Time

The authorization-propagation analysis that the period's security literature treats as its reference framing identifies transitive delegation, aggregation inference, and temporal validity as exactly the axes where classical RBAC and ABAC fail for agentic systems[1][23]. Fan-out makes the problem quantitative: an orchestrator spawning hundreds of subagents[1] is minting hundreds of authorization contexts per tasking. Consequence: a credentialing service for non-human identities is first-class infrastructure (Section 04), aligned with zero-trust doctrine's per-request access evaluation[37], and revocation is designed to cascade through delegation trees.

P5 — Evidence Over Narrative

A monitor reading chain-of-thought caught 95% of malicious behavior until the traces were laundered; then it caught one in ten[22]. Consequence: the observability plane records what the system did, at interfaces the model does not control, and the confirmation gates of Section 04 present operators with evidence bundles rather than model explanations.

P6 — Denial by Default

Decentralized swarm defense with no GPS and no inter-agent communications, validated on real robots, establishes feasibility[28]; the environmental analysis of Section 02 establishes necessity. Consequence: Section 07 specifies the local autonomy envelope — mission intent, engagement constraints, geofences, resource limits, abort conditions, and safe states, encoded before deployment and enforceable locally — and the reconciliation protocol for reconnection.

P7 — Mission Measures Over Model Measures

In a full-stack LEO SATCOM simulation, forecast-based planning cut delay by roughly 10–40% under some load conditions — yet a 14–16% difference in forecasting error yielded less than 1% delay improvement and nearly identical jitter[30]. Better model error does not equal better mission. The complementary lesson from BRIDGE is that the AI component is itself an attack surface whose robustness is a mission property, evaluated there under gradient-based adversarial perturbations[29]. Consequence: Section 08 makes metric traceability a deliverable, and the maturity model conditions advancement on mission-measure evidence.

P8 — Governed Capability Tiers

The industry converged on this unprompted — restricted tiers, qualified-access programs, and security-tuned variants all appeared within one quarter[1] — and open task-to-model routing infrastructure now exists to implement it. Domain benchmarks reinforce the corollary: general-purpose capability does not imply mission adequacy[20], so tier assignment must rest on mission-specific evaluation, not leaderboard rank. Consequence: routing-as-governance is specified in Sections 04 and 08: the router enforces tier policy, budget, and data-domain constraints simultaneously.

The Reference Architecture

The architecture comprises three operational planes, stacked in the direction of tasking flow, crossed by three governing planes that touch every operational component. The diagram shows the arrangement; the subsections that follow specify each plane's responsibilities, components, and failure-containment role. The section closes with a worked trace — the life of a single tasking — showing every plane in action.

Mission & Command Plane

The top plane is where human authority enters and remains. Its components: an intent capture interface that turns an operator's tasking into a structured statement of objective, constraints, geography, time bounds, classification, and resource ceiling; a policy compiler that renders command intent, ROE, and standing orders into the machine-enforceable artifacts the governance plane consumes (permitted tool classes, data domains, action ceilings, confirmation thresholds, delegation depth); and confirmation gates at which designated high-impact action classes pause for human decision.

Two design rules distinguish this plane from a chat front end. First, the confirmation gate presents an evidence bundle — the proposed action, the policy evaluation that admitted it this far, the provenance of the inputs that motivated it, and its predicted effects — not the model's persuasive narrative (P5). Second, decision-support outputs are marked advisory by construction: the plane offers no pathway by which a model recommendation becomes an executed order without crossing a gate bound to an accountable human role (P2). For systems adjacent to use of force, gate placement is where DoDD 3000.09's "appropriate levels of human judgment" is implemented as engineering rather than aspiration[35].

Orchestration Plane

The orchestration plane turns one tasking into governed many-agent work. Its planner/decomposer produces a task graph; its router assigns each node to the smallest adequate model consistent with tier policy (P8), data-domain constraints, and budget; its fan-out controller bounds concurrency, delegation depth, and aggregate resource consumption.

The router deserves emphasis because the industry has already built its commercial analogue: open-source infrastructure whose entire function is directing tasks to suitable models[1]. In a mission system the router is not a cost optimization bolted on afterward — it is a policy enforcement point. A routing decision simultaneously answers: which model tier is this task cleared for, which enclave may execute it, what budget may it consume, and what is the fallback if the preferred model is unavailable or its price has moved (Section 08). Plans emitted by the planner are themselves proposals: they are validated against the mission envelope before any node executes (P3), which is where plan-level hazards — objectives smuggled in through retrieved content, subtasks exceeding the envelope — are caught at one chokepoint instead of N runtime surfaces[23].

Agent Runtime Plane

The runtime plane hosts the model population and the machinery of action: heterogeneous models (frontier API models, domain-adapted open-weight models in classified enclaves, small specialist models at the edge); tool brokers that mediate every tool with per-tool least privilege, argument validation, and budgets; and sandboxed execution for anything that runs code or touches files, with effects captured rather than applied directly. Three commitments define the plane: tool access is governed, not default-on — gating decisions weigh necessity, utility, and affordability, and the tool-use tax is measured in T&E rather than assumed away[2][3]; tool results are untrusted inputs sanitized on the way back in (P3); and no runtime component holds standing credentials — everything an agent can do, it does under the task-scoped identity the governance plane issued for this tasking (P4).

Governance Plane — The Differentiator

The governance plane is the machinery that makes autonomy governed rather than merely instructed. Five services:

• Identity and credentialing for non-human agents. Every agent instance receives a cryptographic identity bound to the tasking that justified it, the capabilities and data scopes granted, a delegation-depth bound, and a temporal validity window. Spawning a subagent narrows scope by construction — delegation never expands privilege — and credentials expire with the task, not the session (P4). This is zero-trust's per-request access decision[37] extended to a population of ephemeral machine identities.
• Policy decision and enforcement points. Policy evaluation (PDP) is separated from enforcement (PEP), so that policy is authored, versioned, and audited centrally while enforcement sits inline at every effectful interface: tool brokers, memory writes, message buses, actuator gateways. Policies are deterministic artifacts under configuration control — reviewable before deployment, diffable across revisions.
• Admission control — agentic transaction processing. The heart of P1. Every proposed action becomes an Action Proposal Record (Appendix D) evaluated against current policy and the effective state — the deterministic projection of what has actually been committed, not what agents believe. Admitted actions produce Commitment Records; the transition log is append-only; multi-step operations register dependency-safe compensation so partial failures unwind cleanly. Mnemosyne demonstrates the discipline is affordable — safety properties enforced against a declared constraint set at under 6% overhead[4] — which removes the standing excuse that deterministic admission is too slow for agentic workloads.
• Action signing. Committed actions are signed by the admitting authority, binding action, credential, policy version, and log position into a verifiable chain. Downstream systems can refuse unsigned effects, which turns the governance plane from a convention into a load-bearing boundary.
• Revocation. Credentials, delegations, and standing permissions are revocable mid-task, and revocation cascades through the delegation tree. Combined with temporal validity, this bounds the blast radius of a compromised or misbehaving agent to a scope and a window.

Memory & Knowledge Plane

Memory is treated as a governed subsystem with its own admission discipline, because ungoverned memory is an error amplifier[7]. Four mechanisms: provenance tagging on every entry (origin, authoring agent and credential, grounding sources, confidence, classification); write admission — memory writes are proposals like any other action, subject to policy on what may persist where; grading audit — self-assessed rewards and confidence scores are routed through de-inflation before they influence retrieval priority, honoring the error-independence requirement that self-grading violates[7]; and grounding verification — claims destined for reports or decisions are checked span-by-span against their cited sources, using the inspection tooling the quarter produced[16][15], with the RAG pipeline itself subjected to systematic generated-test campaigns rather than anecdotal spot checks[17]. Organizational memory — conventions, prior decisions, workflow knowledge persisting across sessions and agents[8][6] — lives here too, under the same provenance and admission rules, partitioned by classification and need-to-know like any other data store.

Observability & Audit Plane

The observability plane records what the system did, at interfaces the models do not control: tool invocations and their arguments, network flows, file and state changes, actuator commands, message-bus traffic, credential issuance and use, policy evaluations, admissions and rejections. The log is immutable and cryptographically chained; the evidence it holds is the material of incident response, T&E, and continuous authorization alike (Section 06). Two rules keep the plane honest: reasoning traces are stored — they are valuable forensic context — but nothing downstream treats them as telemetry or as a detection signal of record (P5)[22]; and generated artifacts leaving the system carry provenance marks (signatures, watermarks where the medium supports them) with a verification path, mirroring the industry's turn toward provenance as product infrastructure[1].

The Life of a Tasking

Governance Mechanisms in Depth

Four mechanism families do most of the governance plane's work in practice. This section specifies each at the level an implementing engineer needs.

Tool Governance

Tool access is a governed resource with a decision procedure, not a capability list. Three findings shape the design: tool-augmented reasoning can underperform plain reasoning when the calling protocol's costs outweigh execution's benefits[2]; models misjudge their own tool need, particularly under budget pressure, while lightweight estimators over hidden states predict true need better than self-reports[3]; and even beneficial tool access can cause regressions on items a model previously answered correctly[10]. The mechanism follows: per-tool least privilege — each tool exposed to each agent with the narrowest argument space and data scope the subtask requires; gating on necessity × utility × affordability, implemented as policy plus, where available, learned need-estimators that are advisory to the deterministic gate, never a substitute for it; budgets — per-agent and per-tasking ceilings on invocations, tokens, cost, and wall time, enforced at the broker; and measurement — T&E explicitly measures the tool-use tax and the regression profile for each agent–tool pairing (Section 06), so that enabling a tool is an evidenced decision.

Identity and Authorization Propagation

The delegation chain is the attack surface classical access control never modeled. Three rules close its failure axes. Monotone narrowing: a spawned agent's credential is computed as the intersection of its parent's scope and the subtask's requirement — there is no code path that widens scope downward. Aggregation review: because individually innocuous permissions can compose into a sensitive capability, policy evaluation considers the credential set active within a tasking, not each credential in isolation; combinations crossing a declared sensitivity threshold require elevated admission. Temporal validity: credentials carry expiry bound to task lifetime; long-running taskings re-authenticate at declared boundaries; and revocation cascades — revoking a parent invalidates every descendant credential in the delegation tree, with enforcement points honoring revocation before any further admission. Signing binds the chain together: every Commitment Record names the credential under which the action was admitted, so audit can reconstruct not just what happened but under whose delegated authority, to arbitrary depth.

Memory Governance — Who Audits the Grades

Section 04 gave the mechanisms; the governance question is who audits the grades. The Echo Gap result is precise about the failure: self-assessed rewards correlate with the agent's own biases, inflating exactly the episodes that should be down-weighted[7]. The audit therefore cannot be performed by the writing agent or any model sharing its failure modes. Options, in ascending assurance: a distinct auditor model under an independent credential; deterministic consistency checks against committed effective state (did the remembered outcome actually occur, per the transition log?); and human sampling audits for memory classes that feed high-consequence decisions. Retention policy is explicit: memory classes declare classification, need-to-know partition, retention period, and the evidence class required for their entries to influence decisions.

Multi-Agent Coordination Controls

Peer-agent messages are untrusted inputs (P3): they cross the same sanitization and admission boundaries as retrieved content, and inter-agent requests for action are proposals under the requesting agent's credential — an agent cannot launder an out-of-scope action through a peer with broader scope, because admission evaluates the originating authority. For high-consequence outputs, verification is structurally independent: verifier agents run under separate credentials, ideally on different model families, so shared blind spots are less likely to align; where determinism is available (schema validation, simulation replay, constraint checking), it outranks model-based verification. Fan-out is bounded by the orchestration plane, and quorum rules for verifier agreement are policy artifacts, not orchestrator improvisation.

Assurance, T&E & Accreditation

The architecture of Sections 04–05 is a claim; this section is how a program substantiates it. The organizing idea is that evaluation must be shaped like deployment, because the quarter's evidence shows static evaluation systematically overstating fielded performance.

Deployment-Shaped Evaluation

Static question-answering is giving way to evaluation that looks like operations, and mission programs should adopt the shift wholesale. Interactive benchmarks make the system actively query a hidden environment, integrate partial observations, update beliefs, and decide when it knows enough — scoring interaction efficiency, contextual robustness, counterfactual revision, and necessity judgment rather than success rate alone[9]. Simulator-in-the-loop evaluation wires agents to deterministic domain simulators and scores the whole computation chain — input construction, execution, output inspection, answer commitment — and has already surfaced the cautionary pattern that tool access lifts aggregate accuracy while regressing on previously correct items[10]. Market-style stressors with hidden preferences, adaptive competitors, and demand shocks expose the gap between benchmark rank and adversarial performance: the strongest agents captured less than a third of hindsight-optimal profit, and rankings shifted with the scoring lens[11]. The portfolio principle: evaluate the agentic system, in its governance harness, against environments that push back.

Adaptive Red Teaming as a Requirement

Trident is the template and the warning. Red teams must learn during the campaign: trainable adversaries that adapt to the deployed defense, probe the governance plane itself, exploit defensive tooling, vary tactics to defeat fingerprinting, sustain long-horizon pressure, and operate across coordinated identities[26][24][25]. Reporting discipline matters as much as the red team: programs should require worst-case degradation and recovery curves, not average reward against fixed scripts — the difference between the two is where the 522% result lived[26]. Learned monitors and guard models are in scope as targets, not just protections[22][21].

Mission-Specific Benchmarks

General leaderboards do not transfer: 52 models managed only moderate zero-shot performance on a military remote-sensing benchmark, weakest exactly where ISR needs them strongest[20], and controlled-dataset results such as MSTAR-based SAR interpretation carry explicit non-extrapolation caveats[19]. Programs should build and control their own evaluation sets — rare targets, sensor phenomenologies, seasonal and geographic variation, decoys, camouflage (including machine-eye camouflage[21]), damage states, degraded imagery — and gate perception claims on them. Domain-adapted open-weight models, evaluable and tunable inside classified enclaves, pair naturally with this discipline (P8).

Reproducible Testbeds and Regression Control

Assurance evidence must be replayable. Deterministic, replay-validated environments that log agent state, decisions, communications, tool usage, and outcomes make results comparable across model versions and vendors[32] — the alternative, anecdotal chat demonstrations, produces evidence that cannot survive an accreditation review. For the model-swap problem specifically, reasoning-trajectory coresets estimate full-benchmark performance at a fraction of the cost, functioning as regression tests when a model is upgraded, substituted, or repriced out of its slot[12]. Emerging mechanistic techniques — geometry-aware filtering of fine-tuning data that cut induced misalignment by 34.5% in one study, and representation-level analyses of harmful-behavior directions — are promising as supplementary evidence for model intake, on the strict understanding that they inspect the model, not the mission system, and do not substitute for the external controls this paper specifies[13][14].

Mapping to the Compliance Landscape

The architecture is designed to make existing compliance regimes easier, not to add a parallel one. Against the NIST AI Risk Management Framework, the governance plane and tier policy implement Govern; the threat model and metric traceability implement Map; the evaluation regime implements Measure; and admission control, revocation, and continuous monitoring implement Manage[33] (mapping table in Appendix B). Against the Risk Management Framework, the observability plane's evidence chain is the continuously generated artifact base that continuous-authorization approaches require[34]. Against DoDD 3000.09, gate placement and advisory-by-construction decision support are the implementable form of appropriate human judgment[35]; the Department's Responsible AI pathway supplies the process scaffolding this architecture gives technical teeth[36]; and adversarial T&E draws its threat taxonomy from the maintained public knowledge base for adversarial machine learning[38].

Contested & Denied Operations

Denial is the design condition, not the exception (P6). The mechanism is the local autonomy envelope: a pre-authorized, locally enforceable artifact deployed to edge nodes that encodes mission intent, engagement constraints, geofences, resource limits, abort conditions, and safe-state behaviors. The envelope is compiled and signed by the Mission & Command plane while connectivity exists; at the edge it is enforced by a local governance runtime — a miniature of the admission machinery of Section 04 — that continues to gate agent proposals when the reachback link is gone. Local autonomy widens within the envelope as connectivity narrows; it never widens the envelope itself.

Feasibility is not speculative: decentralized swarm behaviors using only onboard sensing and relative measurements, with no GPS and no inter-agent communications, have been validated on real robots[28]. The architectural corollaries: centralized AI is an enhancement, never a single point of mission failure; degraded-mode transitions are explicit, tested states rather than emergent behavior; and edge nodes run the smallest adequate models under the same credential discipline as the enterprise, sized to disadvantaged compute.

Mission Metrics & Operational Economics

Requirements in Mission Measures

Programs should write agentic-AI requirements in measures of effectiveness — latency, availability, mission completion, resource efficiency, false-alarm burden, operator workload — and require from vendors an explicit trace from every model-level metric to its claimed operational effect (P7). The LEO forecasting result is the template: a 14–16% improvement in model error produced under 1% mission-level improvement and no meaningful jitter change[30]. Once a model is good enough, compute, scalability, robustness, and response time dominate; a ten-percent model-metric gain with no mission-measure effect should move no procurement decision. The trace requirement also disciplines the architecture itself: confirmation gates and admission overheads are costs in operator workload and latency, and they are measured, budgeted, and reported like any other mission property — the sub-6% admission overhead demonstrated by Mnemosyne[4] is the existence proof that governance need not dominate the budget.

Agent Workloads Are Not Chat Workloads

The production evidence is now in — one sampled month of a deployed coding agent[5]:

Mission planners inherit the lesson at design time: capacity, cost, and latency models built on chat assumptions will be wrong in the direction that hurts, and serving infrastructure is part of the mission system's performance case, not a procurement afterthought.

Routing, Price Volatility, and Portability

Model economics moved in-quarter — price increases from one major provider took effect the day this paper is dated, introductory pricing appeared from another, and open routing infrastructure now treats model choice as a per-task decision[1]. Buying intelligence increasingly resembles buying cloud: tiers, volatile rates, and a router in front. The architecture's response is already in place: the router (Section 04) enforces tier and budget policy per task; regression coresets[12] plus the replayable testbeds of Section 06 make substitution a controlled, evidenced event rather than a scramble; and the smallest-adequate-model discipline caps exposure to any single vendor's pricing power. Portability is thereby a governance property with a test suite, not a contract clause.

Reference Implementations

Four vignettes show the architecture instantiated. Each names the planes it exercises hardest, the threats it is built against, and the mission measures that govern it.

SATCOM Resource-Management Copilot

A ground-segment copilot proposes beam scheduling and power allocation for a LEO constellation, in the pattern of digital-twin-coupled schedulers that handle discrete beam selection and continuous power jointly[29]. The scheduler's recommendations are proposals: a deterministic constraint layer — link budgets, interference limits, priority-of-service rules, spectrum authorities — admits or rejects each plan before it reaches the network, and admitted plans are signed and logged. Because the scheduler is an attack surface in contested space operations, its T&E includes gradient-based adversarial perturbation of inputs, following the robustness-evaluation precedent[29], alongside the forecasting discipline of Section 08: model-error improvements are accepted only with demonstrated mission-measure effect[30].

Autonomous Cyber Defense Cell

A defensive cyber agent triages alerts, proposes containment and remediation, and drafts hunt queries. Every action follows propose/decide: deterministic analysis and formal runtime checks gate execution, with an independent action-alignment oracle in the loop, on the CyberLLM pattern whose deterministic layer alone achieved perfect precision on a third of labeled vulnerabilities before LLM passes roughly doubled coverage[27]. Accreditation rests on adaptive red-team campaigns — learning adversaries in the Trident mold, with worst-case degradation and recovery reported[26] — and attribution products are weighted toward infrastructure, provenance, and campaign-level evidence rather than TTP similarity, which synthetic adversaries now erode[25].

ISR Exploitation Assistant

An analyst-facing assistant layers multimodal reasoning above specialized perception networks and retrieval from authoritative intelligence stores — the composite architecture the benchmark evidence favors over any single general-purpose model[20]. Interactive target characterization in the SAR-assistant mold is in scope[19], but every perception capability is gated on program-controlled, mission-specific test sets spanning rare targets, decoys, camouflage — including adversarially optimized physical patterns[21] — and degraded imagery; ensemble diversity, multi-sensor fusion, and uncertainty estimation keep one detector's failure from becoming a system failure. Claims destined for reporting pass span-level grounding verification[16].

C2 and Wargaming Decision Support

A course-of-action assistant generates options, stress-tests them in simulation, and summarizes trade-offs — and is advisory by construction. Escalation controls and use-of-force constraints exist as external policy and authorization mechanisms, never as prompt guidance, because prompting demonstrably fails to constrain escalation behavior[31]; evaluation runs in deterministic, replay-validated environments that log every decision and communication, comparable across model versions and vendors[32]. Research directions in zero-annotation intent inference are relevant to making the assistant better at understanding commander intent under uncertainty[18] — while changing nothing about where authority sits.

Maturity Model & Adoption Roadmap

Autonomy is granted in levels, and each grant is preceded by assurance evidence at the prior level. The table defines four levels; the governing rule is that a system advances when its evidence does — advancement is evidence-gated, not schedule-gated.

Acquisition Implications

Five contract-language consequences follow directly:

1. Write requirements in mission measures with mandatory metric traceability (P7).
2. Require adaptive red-team results with worst-case reporting as a deliverable, not a demonstration.
3. Procure data rights to the evidence chain — logs, credentials, policy versions — because the audit trail is the accreditation artifact.
4. Require vendor-neutral routing and a demonstrated substitution rehearsal, so no model vendor becomes a single point of programmatic failure.
5. Align the authorization strategy with continuous-ATO practice from the start: the observability plane exists to feed it[34][33].

Conclusion: Rapid Experimentation, Bounded Authority

The commercial frontier spent mid-2026 learning to delegate; the research literature spent it demonstrating what delegation costs when an adversary adapts. Both lessons are gifts to the mission-system architect, because together they specify the job: admit the capability, bound the authority. Governed Autonomy names the resulting doctrine — generation probabilistic, commitment deterministic — and this paper has tried to make it buildable: eight principles, six planes, artifact-level mechanisms, an assurance regime shaped like the threat, and a maturity model that ties every grant of autonomy to evidence.

Rapid experimentation, bounded authority.

Appendices: Traceability, Mappings & Artifacts

Appendix A — Threat-to-Control Traceability Matrix

Every threat and failure mode of Section 02, traced to the principles that answer it, the primary architectural controls, and the assurance evidence that substantiates the control.

Appendix B — Mapping to the NIST AI Risk Management Framework

The architecture against the four AI RMF functions[33].

Appendix C — Glossary

Appendix D — Artifact Schemas

Minimum field sets for the two records at the heart of admission control. Programs extend them; nothing removes a field.

References & Evidence Base

Bracketed numbers throughout the paper link to the entries below. Preprint identifiers are as surveyed in The Frontier Dispatch, Issue 02; standards and directives are cited in their governing versions.

1. Continuum Resources LLC, "The Frontier Dispatch," Vol. I, Issue 02 (Industry Desk, Research Bench, Defence File), May–August 2026.
2. Kaituo Zhang et al., "Are Tools All We Need? Unveiling the Tool-Use Tax in LLM Agents," arXiv:2605.00136, May 2026.
3. Wu et al., "To Call or Not to Call: A Framework to Assess and Optimize LLM Tool Calling," arXiv:2605.00737, May 2026.
4. Chang, Geng, and Chang, "Mnemosyne: Agentic Transaction Processing for Validating and Repairing AI-generated Workflows," arXiv:2607.00269, July 2026.
5. Liu et al., "Agentic Coding in the Wild: Characterizing GitHub Copilot Traces at Production Scale," arXiv:2608.00101, August 2026.
6. Talebirad et al., "From Signals to Structure: How Memory Architecture Drives Language Emergence in LLM Agents," arXiv:2607.00233, July 2026.
7. Asadolahi et al., "Memory Reward Inflation in Self-Improving LLM Agents," arXiv:2608.00017, August 2026.
8. Dhanyamraju and Raghav, "Shared Organizational Memory for Enterprise Coding Agents: System Design and Deployment Snapshot," arXiv:2608.00122, August 2026.
9. Fan et al., "Evaluating Interactive Reasoning in Large Language Models: A Hierarchical Benchmark with Executable Games," arXiv:2606.00103, June 2026.
10. Ke Zhang et al., "PHREEQC-MCQ-200: A Diagnostic Benchmark for Tool-Augmented Scientific Simulator Agents," arXiv:2607.00436, July 2026.
11. Ahmed, Cai, Minaei, and Rachuri, "Can LLM Agents Price Competitively? A Dynamic Multi-Attribute Auction Benchmark for Agentic Commerce," arXiv:2608.00102, August 2026.
12. Pan et al., "CoT-Core: Accelerating LLM Evaluation via CoT-Aware Coreset Selection," arXiv:2608.00014, August 2026.
13. Minegishi et al., "Understanding Emergent Misalignment via Feature Superposition Geometry," arXiv:2605.00842, May 2026 (ACL 2026).
14. Chua et al., "HARC: Coupling Harmfulness and Refusal Directions for Robust Safety Alignment," arXiv:2607.00572, July 2026.
15. Zhao et al., "TIGER: Traceable Inference with Graph-Based Evidence Routing for Mitigating Hallucinations in Multimodal Generation," arXiv:2606.00232, June 2026.
16. Belikova et al., "SIRIN: A Unified Toolkit for Detecting Contextual Hallucinations in Retrieval-Augmented and Memory-Grounded LLM Systems," arXiv:2608.00033, August 2026.
17. Maiztegi, Ayerdi, Illarramendi, and Arrieta, "RAG-TESTER: Automated End-to-End Testing of Retrieval-Augmented Large Language Models," arXiv:2608.00054, August 2026.
18. Shunchi Zhang et al., "MindZero: Learning Online Mental Reasoning With Zero Annotations," arXiv:2606.00240, June 2026 (ICML 2026).
19. Ramirez et al., "Towards a Large Language-Vision Question Answering Model for MSTAR Automatic Target Recognition," arXiv:2605.10772, May 2026.
20. Lai et al., "RRS-10K: A Multitask Vision-Language Model Benchmark for Rare Remote Sensing Image Interpretation," arXiv:2607.24810, July 2026.
21. Bayer et al., "Structured Adversarial Camouflage via Voronoi Diagrams," arXiv:2606.17711, June 2026.
22. Shiromani and Richter, "A False Average: Chain-of-Thought Monitors Collapse Where They Are the Only Defense," arXiv:2608.00583, August 2026.
23. Xiao, Wang, and Sun, "S³: Improving Agent Safety through Multi-Stage Defense," arXiv:2608.02683, August 2026.
24. Schwarzer et al., "AI Model Extraction Attacks: Bypassing Single-Client Assumptions in Defenses," arXiv:2606.03381, June 2026.
25. Balassone et al., "Synthetic APTs: the Collapse of TTP-Based Attribution," arXiv:2606.07158, June 2026.
26. Masukawa et al., "Trident : How to Break Deep Reinforcement Learning Cyber Defenses (Agentic)," arXiv:2608.04317, August 2026.
27. Petrovic et al., "CyberLLM: A Multi-Agent LLM Framework for Autonomous Detection and Guarded Response in Automotive Cybersecurity," arXiv:2608.06651, August 2026.
28. Silveria et al., "Decentralized UAV Swarms for Ground Target Protection in GPS- and Communication-Denied Environments," arXiv:2607.20710, July 2026.
29. Zheng et al., "Beam Hopping Low Earth Orbit Satellite Resource Allocation for Differentiated Services and Robustness Analysis under Model Attacks," arXiv:2607.03859, July 2026.
30. Demirci et al., "The Impact of Demand Forecasting on Delay and Jitter in DVB-Based Beam-Hopping LEO Networks," arXiv:2608.07901, August 2026.
31. Chen et al., "To Nuke or Not to Nuke: LLMs' (Missing) Ethical Reasoning and Actions in a High-Stakes Decision-Making Simulation," arXiv:2606.08310, June 2026.
32. Matlin, Song, Zang, and Riedl, "No One Wins in Nuclear War: A Social Simulation of Military Decision-making," arXiv:2608.01868, August 2026 (Social Sim Workshop, COLM 2026).
33. National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, January 2023.
34. National Institute of Standards and Technology, Risk Management Framework for Information Systems and Organizations: A System Life Cycle Approach for Security and Privacy, NIST SP 800-37 Rev. 2, December 2018.
35. U.S. Department of Defense, DoD Directive 3000.09: Autonomy in Weapon Systems, updated 25 January 2023.
36. U.S. Department of Defense, Responsible Artificial Intelligence Strategy and Implementation Pathway, June 2022.
37. U.S. Department of Defense, DoD Zero Trust Strategy, November 2022.
38. MITRE Corporation, ATLAS: Adversarial Threat Landscape for Artificial-Intelligence Systems, knowledge base, continuously updated.