You Can't Pin an Agent in YAML: Agent "Infrastructure as Code" Promises Reproducibility That Doesn't Exist — Codify the Boundaries, Not the Brain

The result, up front

Before I argue with anyone, here is the practical take you can act on today:

  • Codify, in versioned YAML under GitOps:** the tool allowlist with per-tool least-privilege scopes, short-lived scoped credentials, the policy stanzas (what the agent may act on, when it needs a human), the model pins and eval/drift checks, and the rollout logic. These are things you can diff, audit, and roll back.
  • Accept, and stop trying to lock:** the reasoning path, the tool-call sequence, the exact output. You never could predict those anyway, and every attempt to pin them is theater that buys you silent corruption.

Agent "infrastructure as code" — the movement that says "declare your agents in YAML and treat them like Terraform" — is real, it's shipping now, and it's not what it claims to be. It is a governance system for the boundary around an unpredictable brain. It is not a reproducibility system for the brain itself. Build the first. Stop buying the second.

Here is why the difference is not pedantry, and how to apply the split to a fleet you actually run.

---

The claim everyone is repeating

A standardization wave is consolidating right now, and it is not a fringe idea:

  • On December 9, 2025, the Linux Foundation formed the Agentic AI Foundation (AAIF), taking in OpenAI's AGENTS.md convention — by then already adopted by more than 60,000 open source projects and agent frameworks, including Codex, Cursor, Gemini CLI, GitHub Copilot, Jules, and VS Code — plus Anthropic's MCP and Block's goose (Linux Foundation press release).
  • New orchestration runtimes are shipping on the exact promise you'd expect. Orloj, a SHOW HN with an Apache-2.0 runtime on GitHub, literally frames its pitch as "Agents are infrastructure" and "It's Kubernetes for AI agents... the missing infrastructure layer between demo agent and production agent fleet" — declare agents, tools, and policies as YAML, then schedule, execute, and govern them (Show HN: Orloj, GitHub OrlojHQ/orloj, orloj.dev).
  • A parallel tooling wave sells the same reproducibility hook. Archon (~23k stars) brands itself as bringing "infrastructure-as-code discipline to AI-assisted development," freezing model version/temperature/seed in YAML and using content-addressed hashes for file operations to tame LLM non-determinism through configuration (Archon analysis).

So the message the builder is hearing, more or less explicitly, is: declare your agents in YAML, diff them, roll them back, reproduce them — just like you do with cloud infrastructure.

That last word — reproduce — is the load-bearing promise of the entire category. It is also the part that does not exist.

---

What the evidence actually says

Start with why cloud IaC works at all, because that's the model the sales pitch borrows.

Reproducibility never came from the YAML. It came from the executor. Terraform gives you reproducible infrastructure because two things are true at once: you declare a desired state, and a deterministic, idempotent executor applies it. Run the same plan against the same state and you get the same result, every time. The declarative file is the input; the deterministic apply is what makes the output repeatable.

Now swap the executor for an LLM. The same declarative input no longer yields a point — it yields a distribution of behaviors, because a token sampler is probabilistic by construction. Config can pin what an agent may do (permissions, tool scope, policy). It cannot pin how it reasons. This is the mechanistic reason agent-IaC is a governance system, not a reproducibility system. (Microsoft's "Platform Engineering for the Agentic AI era" reaches the same conclusion from the platform side: IaC stays the system of record because it gives a deterministic desired state, versioned history, reviewable plans, and drift reconciliation — and agents operate through IaC; they do not replace it.)

The uncomfortable part is that the naive implementation — letting the model write the config — is not just philosophically wrong; it's measurably dangerous. A 2026 arXiv benchmark, "Don't Let the Model Write the YAML" (Daveneni, arXiv:2609.00227), ran 83 real Kubernetes field-change tasks through three ways of applying an agent's idea about a config edit:

  • Full-file rewrite is capability-dependent and unsafe. A small model that reflowed the whole manifest to change one field got it right 2.4% of the time and altered an unrelated line in 97.6%** of outputs, with a mean diff of 921 lines. The file it "fixed" was effectively destroyed.
  • A frontier model is better but non-deterministic. Claude Sonnet 5 rewrote faithfully most of the time (97.6% correct, near-minimal diff), but on 6 of 83 tasks (7.2%)** it produced a correct edit on some seeds and a silently corrupted one — a dropped field, a collateral change — on others. For a change that gets committed and shipped, non-deterministic silent corruption is disqualifying.
  • A tolerant unified-diff path still leaks. Applying diffs with tolerant patching worked for ~96% of diff applications yet silently misapplied ~1 in 7 (14–20%)**, landing the change wrong with no error signal at all.

Frame these numbers honestly: they're one benchmark, on one corpus, from mid-2026 — evidence of the mechanism, not a permanent benchmark. But the mechanism is the whole story. The executor that agent-IaC trusts to "apply the fix" is the very thing that makes the system unreproducible. An LLM is not the deterministic, idempotent engine Terraform's reproducibility rests on.

The adoption figures deserve the same honesty. The "60,000+ projects use AGENTS.md" count measures repositories that include the file and tools that read it — format adoption, not governed fleets. Format adoption is not evidence of governance-in-practice, and treating it as such is how the hype outruns the mechanism. (Linux Foundation AAIF press release states the adoption number; the interpretation that it proves governed fleets is our inference, and it does not hold.)

---

Where the claim is roughly right

Do not throw the whole idea out. Agent-IaC genuinely delivers the parts that used to be impossible to touch, and those are worth a lot:

  • Versioned manifests and an audit trail.** You can now answer "what is my fleet allowed to do, and when did that change?" from a PR review, not from memory.
  • Rollback of policy and permissions.** You can revert a permissions change that broke something. That is real, and it is new.
  • Per-tool least-privilege scopes you can diff.** This is the single most valuable thing the manifest gives you, and most of what follows is about it.
  • Drift detection.** The manifest can tell you when the live environment no longer matches the declared boundary.

None of these promise reproducible reasoning. They promise a controllable, auditable, reversible boundary around an unpredictable brain. That is the genuine win, and it is completely different from the sales pitch.

---

Where it falls apart

The claim falls apart in exactly one place, and it is the place the marketing points to as the centerpiece: reproducibility.

The evidence above is the sketch of why. The reject_if_weak bar for this piece was: don't collapse into "buy a self-hosted agent-IaC product," and don't write adoption porn with no central claim. The central claim stands on the mechanism: you can codify what an agent may touch; you cannot codify how it reasons. The promise of reproducible agents is unimplementable because the executor is non-deterministic — and the natural attempt to make it work, letting the model author the config bytes, produces measured silent corruption.

Also note the boundary on the boundary. The data on reproducible or even reliably predictable agent behavior shows the reverse of the promise (the LLM is a non-deterministic executor), and no source demonstrates a team achieving behavioral reproducibility of an agent. (arXiv:2609.00227 measured the corruption; the absence of any demonstrated reproducible fleet is the absence we're building the "accept" column on.)

---

The better version of the claim

Here is the claim stated the way it actually survives contact with evidence:

Don't try to Terraform an agent's reasoning — you can't, and the attempt (letting the model write the config) is measured and unsafe. Do codify the boundaries: manifests, per-tool least-privilege scopes, policy, guardrails, evals, drift detection — versioned and PR-reviewed under GitOps, with a deterministic apply contract between the model's intent and the actual mutation. Codify the box, never the brain.

That claim has two moving parts worth unpacking, because they're the pieces you'll rebuild elsewhere.

Part 1: The deterministic apply contract (model proposes, machinery disposes)

The fix the benchmark itself lands on is the cleanest transferable move in this whole story. When an agent wants to change a config value, do not let it rewrite the file, emit a diff, or re-serialize the document. Instead, split the two jobs by what each side is good at:

  • The model generates intent.* It emits a structured field change*: resource, field, new value. Not bytes. Not a reflowed document.
  • Deterministic code performs the mutation.** A parser locates the exact character span of that field in the raw file and edits only those bytes. No re-serialization, no collateral reflow, no dropped fields.

Model proposes; machinery disposes. The model is good at the decision and bad at determinism; code is good at determinism and bad at the decision. Put each where it wins. (arXiv:2609.00227 is the source for the deterministic span-edit apply contract.)

Where this bites in practice: the task that destroys your manifest is never "add a node" — it's "change this one field's value on line 312 of a 900-line spec," where the difference between right and wrong is whether the tool touched only that field or re-serialized the whole object. With the apply contract, the consequence of a bad model intent is bounded and visible: it names a field value, and a validator can reject it deterministically. The consequence of letting the model write the config is a 921-line diff you might not notice until a dependency silently breaks.

Part 2: The blast-radius reframe (least agency)

The defining risk of an autonomous agent is its blast radius, and it is governed at the boundary, not the reasoning. OWASP's Top 10 for Agentic Applications 2026 — released December 9, 2025 — catalogs ASI01–ASI10, including Agent Goal Hijack (EchoLeak, CVE-2025-32711, a zero-click indirect-prompt-injection exfiltration against Microsoft 365 Copilot), Tool Misuse & Exploitation (ASI02), Identity & Privilege Abuse (ASI03), and Rogue Agents (ASI10). Its core defensive principle is least agency: grant each agent only the minimum autonomy, tools, and permissions its task requires — expressed per-tool — so a hijacked agent cannot reach what it does not hold. (OWASP Top 10 for Agentic Applications 2026)

You cannot predict an agent's steps, which is exactly why you enumerate and constrain what it may touch. As the Cycode framing of the OWASP model puts it: an agent's blast radius equals the union of every permission it holds, and most agents today hold far more than their task requires. (That framing is vendor marketing, but the underlying least-agency principle is the peer-reviewed OWASP framework and the incidents are disclosed.)

Push the framing to its consequence: the manifest is a least-privilege policy file, not a reproducibility spec. And that's why agent-IaC is worth doing at all — a versioned, PR-reviewed, diffable permission boundary is the one thing in the stack that rollbacks, audit trails, and drift detection actually apply to.

---

The reusable object: codify vs. accept

Here is the decision rule that applies to any agent stack, from Orloj-style orchestration to a hand-rolled fleet:

| Codify in versioned YAML under GitOps | Accept as emergent — do not try to lock |

|---|---|

| Tool allowlists, with per-tool least-privilege scopes | The reasoning path and decision sequence |

| Short-lived, scoped credentials; per-agent identity | The exact tool-call order |

| Policy stanzas: what it may act on, what needs human confirmation | The exact output tokens on any run |

| Model pins (version/temp/seed) + eval & drift checks | "Reproducibility" of agent behavior |

| Rollout logic (canary, kill switch, rollback) | Any guarantee of how it reasons |

Rule of thumb when you see a fight coming: if you can diff it, audit it, and roll it back, codify it. If you could not predict it even in the best case, accept it. Would you ever have been able to reproduce a specific reasoning trace? No. So do not spend your governance budget trying.

The minimal starter manifest

A usable seed, not a product. Every line maps to a codify-side row above:

`yaml

agent: support-triage

model:

pin: anthropic/claude-sonnet-5 # model pin — not a reproducibility promise, just a stable target for evals

identity:

credential: short-lived-scoped # per-agent identity, rotation, no standing keys

tools: # least-agency allowlist, per-tool scope, deny by default

  • name: jira.search

scope: "readonly, repo:support" # least privilege per tool

  • name: slack.respond

scope: "thread-only, human-confirm" # ASI09-style human-confirmation gate

policy:

act_on: "jira:SUPPORT-*, slack:threads in #support"

needs_human: "any external send, any write outside jira"

deny_egress: true # OWASP least-agency / blast-radius

eval:

drift_check: on-apply # manifest vs. live boundary, per apply

model_eval: pin-version-batch # regression gate before rollout

rollout:

canary: true

kill_switch: webhook-or-cli-now

`

And the apply rule pasted at the top, because it governs how anything in this file gets changed:

The model emits a structured field-change intent. Deterministic code performs the mutation. Never let the model author the config bytes, because silent corruption is the cost of reproducibility theater.

Where the reader gets stuck (and the way through)

  • "But shouldn't the model write the YAML for me?"* Only if you save more than the corruption costs you. The benchmark's numbers (2.4% small-model correctness; frontier-model silent corruption on ~7% of tasks) are the price of letting the model author bytes. Keep the model on generating intent*; let deterministic code own the mutation. (arXiv:2609.00227)
  • "What about my framework, it handles this?"** AGENTS.md adoption counts format adoption, not governance-in-use. The manifest conventions you're choosing between (AGENTS.md, Orloj YAML, Archon's config) are competing formats; which one wins matters less than whether you've separated the deterministic apply from the model-authored bytes.
  • "I don't manage a fleet, I have two agents."** The split applies at any scale. The blast radius of one agent with a wide credential is exactly the union of what it holds (least agency, OWASP). Two agents with per-tool scopes are safer than one agent with a standing production key.

The compressed ending

Cloud IaC became reproducible because of the executor — a deterministic, idempotent engine that always did the same thing with the same input. An LLM is not that engine. The agent-IaC movement adopts the declarative YAML and quietly imports the executor's job onto a probabilistic token sampler, and the moment you trust the model to author the config, you trade "reproducible" for "silently corrupted" — measured, not hypothetical.

So stop buying reproducibility. Codify the box: the tool allowlist, the least-privilege scopes, the short-lived credentials, the policy, the evals, the rollout — versioned, PR-reviewed, rollback-able, drift-checked. Accept the brain: the reasoning, the tool calls, the output. You could never reproduce those anyway.

And whatever gets changed, let the model propose the intent and let deterministic code perform the mutation. Model proposes; machinery disposes. Codify the box, never the brain.

That is the difference between an auditable, least-privilege fleet and an ungoverned one — and it is there on purpose.

---

Sources

  • Show HN: Orloj – agent infrastructure as code (YAML and GitOps): <https://news.ycombinator.com/item?id=47526813>
  • Orloj orchestration runtime (GitHub): <https://github.com/OrlojHQ/orloj>
  • Orloj site ("It's Kubernetes for AI agents..."): <https://www.orloj.dev/>
  • Linux Foundation: formation of the Agentic AI Foundation (AAIF), AGENTS.md/MCP/goose, 60,000+ adoption: <https://www.linuxfoundation.org/press/linux-foundation-announces-the-formation-of-the-agentic-ai-foundation>
  • Microsoft "Platform Engineering for the Agentic AI era" (agents operate through IaC, not replace it): <https://devblogs.microsoft.com/all-things-azure/platform-engineering-for-the-agentic-ai-era>
  • OWASP Top 10 for Agentic Applications 2026 (ASI01–ASI10, least agency): <https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/>
  • Davineni, "Don't Let the Model Write the YAML" (arXiv:2609.00227, Aug 2026): <https://arxiv.org/html/2609.00227>
  • Archon / coleam00-Archon (23k stars), secondary source on deterministic workflows: <https://aisignal.dev/analysis/coleam00-archon>

---

X entryway list (from the approved angle)

Distribution entryways named in the angle brief. Hooks are not yet written — this names each post's type, its one claim/mechanism, and whether it links.

  • Hook: sharp claim** — Agent "infrastructure as code" promises the one thing Terraform made famous and structurally cannot deliver: reproducibility (deterministic executor vs. non-deterministic LLM). You can codify what an agent may touch; you cannot codify how it reasons. Agent-IaC is a governance system, not a reproducibility system. → Links to article (link is the non-deterministic-executor mechanism; the inversion stands alone).
  • Hook: surprising proof point** — Letting an LLM edit a Kubernetes config to change one field: small model 2.4% correct, frontier model non-deterministic (silent corruption on some runs, 6 of 83 tasks), unified diff silently misapplied ~1 in 7. The model agent-IaC trusts to "apply the fix" is the exact thing that makes it unreproducible. → Links to article (measured failure is the entry; article explains the deterministic apply contract).
  • Hook: mechanism** — Split "generate intent" from "perform mutation": LLM emits a structured field change, deterministic code locates the span and edits the bytes — no re-serialization, no model-written YAML. Model proposes, machinery disposes. → Links to article (the contract stands alone; article is the full decision rule).
  • Hook: workflow / build sequence** — Treat the agent manifest like a least-privilege policy, not a reproducibility spec: per-tool scopes, short-lived credentials, policy stanzas, model pins, drift checks, versioned and PR-reviewed under GitOps. OWASP: blast radius = union of every permission an agent holds. → Links to article (governance workflow is the actionable unit; article is why boundaries, not reasoning).
  • Hook: result** — The one decision rule for agent fleets: codify the box (tools, permissions, policy, evals — things you can diff, audit, roll back), accept the brain (reasoning, tool-call order, output — you never could predict it). Founders chasing reproducible agents over-engineer; founders who govern boundaries get an auditable fleet. → Links to article (the rule stands alone; article is the manifest starter that implements it).

Note on distribution: per the repurposing rule, each post carries its own reason to exist (sharp claim / proof / mechanism / workflow / result) and the article link is a bonus, not the whole point. No post is a shortened version of the article with the same hook.