LLM is a component, not the pilot
It responds within stages; it never controls the flow of control.
Genetic Engineering
Video opens in “magic”: a free AI agent solves a task in seconds and delights the audience. Then the same demo is brought to production and it's undone — unpredictable loops, cost that explodes and hallucinations that propagate, exactly the side Chaotic from the storyboard next door.
The narrative then turns the key to the engineering: the same problem, now solved by a deterministic pipeline where LLM only enters the path where it is irreplaceable. Sequence steps, contracts bar the error, linear cost and validated and explicable outputs — the side Disciplined.
Closing message for PMs and executives: the capacity comes from the model; trust comes from engineering. No need for less ambition, But less AI on the critical path.
Capacity comes from the model; trust comes from engineering. An interactive journey by principles that separate a charming prototype from a product that supports production.
You prefer the essentials first? Read synthesis (~1 min)
How it works
Information flows through deterministic steps; the LLM enters only where it is irreplaceable, and the output returns validated and explained.
The 10 principles
The thesis in ten rules. Reliable AI systems are born of disciplined architecture — not from Smarter promps.
It responds within stages; it never controls the flow of control.
Same entry, same exit — always. Non-determinism is flawed.
Sensitive motor and exposed interface, connected by a narrow contract.
Solve with rules and data; spend AI only on interpretive residue.
Model output is unreliable input until validated.
Evidence
Absence of data is explicit uncertainty, never "false".
Quota and cost are domain status; process in waves, never pay twice.
BDD with degraded scenarios, DoR/DoD rigid, honest blocking.
Drivers, gaps and provenance build the trust that generates adoption.
Five pillars
The ten principles answer how build. The five pillars respond how much enforceability and when. The premise does not change: LLM is a component, never the pilot — discipline is what makes autonomy reliable.
Autonomy is not a switch (trust/do not trust) — it is a continuous. The correct position of a task in this continuum is the function of two variables: error cost (from cosmetic to catastrophic) and reversibility (from instantaneous to irreversible).
Analogy P&D lab and certified production line. The mistake is not having both modes — it is move an artifact between zones without reprocessing it under destination zone rules.
Connect with P1 · component, P4 · deterministic-first. Use Autonomy Calibrator to position a task in the matrix.
Well placed restrictions do not slow down the system — they increase maximum safe speedBecause they reduce the space of states the agent needs to reason on. Three instruments form the sandbox:
Analogy An F1 is not fast though carbon brakes — is fast because of them: the confidence in braking is what allows to attack the curve later. Remove the brakes does not accelerate the car; prevents it from completing a turn.
Connect with P5 · contracts, P9 · poor-closed governance.
Reasoning components, tools and interface communicate by explicit and versatile contractsNever by direct coupling. Two borders matter:
Declared limitation — "A2UI"
"A2UI" is treated here as architectural pattern (the agent ↔interface border), not as canonical industry specification. Adjacent royal protocols include AG-UI (Agent-User Interaction) and A2A (Agent-to-Agent, Google). The MCP is open protocol with public specification. Validate the nomenclature before publishing it as an established technical term.
Connect with P3 · Brain × display, P5 · contracts, Agent Ready · publish the MCP border on the Web.
One Intentional System has the behavior governed by an explicit architectural intention — not by the statistically more likely output. The journey moves the source of the guarantee from a place that is not controlled (the internal distribution of the model) to one that is controlled (the architecture around it).
Analogy The noisy analog component inside a digital circuit. No one "fixes" sensor noise — encapsulates the sensor in an architecture that extracts the signal and discards the rest.
Connect with P2 · determinism, P6 · epistemic layers, P8 · ledgers. Diagnose your internship in maturity model Down.
Heavy discipline — DAGs, rigid contracts, Ledgers, ADRs — has cost of setup and cost of stiffness. These costs are only paid when amortised by longevity, scale or criticality. Where there is nothing to amortize, rigor is negative, not neutral. Two contexts call for deliberate lightness:
Analogy The scaffolding versus the permanent structure. The error is not using scaffolding — it is let the scaffold turn the structure.
Transition rule
Lightness is legitimate while the artifact remains in its original pole. The moment it is promoted — it is spent by users, money or data — the promotion No, no, no. the upgrade of guarantees. The question of governance is not "is this exploitation or production?" but "Does anyone ever depend on this as if it were production?". If so, the permanent structure is not optional.
Connect with P4 · deterministic-first, Journey · knowing when not to build.
The guarantee migrates from the model to the four-stage architecture. Select a stage for see what already exists, what is missing and the next step — a diagnosis of "where I am now".
From abstraction to code
It is not theory: each principle becomes few lines of disciplined engineering. Pure Python, strict contracts and the LLM contained at a single step.
from pydantic import BaseModel, Field
class Inference(BaseModel):
model_config = {"extra": "forbid"} # rejeita campo inesperado do LLM
intent: str
confidence: float = Field(ge=0.0, le=1.0) # limite no schema, nao no if
def run_pipeline(item, llm):
# Serve a qualquer dominio: leads B2B, tickets de suporte,
# pedidos de e-commerce, laudos medicos -- a estrutura e a mesma.
signals = collect_signals(item) # M1: deterministico (APIs, banco, sem IA)
if signals.missing: # Open-World: ausencia = incerteza
return Inference(intent="desconhecido", confidence=0.0)
raw = llm.extract(signals.snippet) # M2: LLM so no residuo interpretativo
return Inference.model_validate(raw) # valida a saida (input nao confiavel)
class TokenLedger:
def __init__(self, daily_cap: int):
self.cap = daily_cap
self.spent = 0
def try_spend(self, n: int) -> bool:
if self.spent + n > self.cap: # teto do dia atingido
return False # falhar e parar: sem fatura surpresa
self.spent += n
return True
ledger = TokenLedger(daily_cap=1000)
for item in fila:
if not ledger.try_spend(cost(item)):
mark_pending(item) # retoma na proxima onda
break # para limpo, em ponto seguro
def explain(score):
# nunca exponha o numero: traduza em "por que agora" + "o que falta"
return {
"why_now": [d.text for d in score.positive_signals],
"gaps": score.missing_signals, # honestidade Open-World
"sources": [d.url for d in score.references],
}
Guided Track
Ten chapters, from prompt chaos to complete design. Open each to see the goal and What you come out with.
No one builds a banking system by letting the database "decide freely" what to do with each transaction. The bank is powerful, but it operates inside of transactions, constraints and rigid contracts — it is powerful Because It's contained. An LLM deserves the same architectural respect.
The problem: the most seductive way to build with AI is to chain prompts and give total autonomy to the model. It works in a demo; in production, it generates defects that classic software rarely faces — infinite loops (one agent returns the task to the other and each turn costs money), unpredictable cost, hallucinations that spread when the invented output of one turns into the other, and irreprobability (the same input produces different outputs, impossible to debug).
The solution is cultural before technique: stop treating the model as magic and treat it as engineering. It applies to the probabilistic component everything we already know about reliable software — interface contracts, deterministic tests, budget control, isolation of responsibilities.
In practice, this means that The ability comes from the model, but trust comes from engineering around it.
Think of a fixed flowchart of a factory: each station does a specific thing and passes on the result. Compare it to a room where several people talk trying to decide what to do next. The first is a DAG — a graph of fixed steps; the second is an open conversation between models.
The problem: giving LLM authority over the flow of control ("agents decide everything from end to end") is flexible and impressive — and impossible to guarantee. You can't promise deadline, estimate cost, or audit.
The solution is to separate two layers that are usually confused. A orchestration layer/I-O can be asynchronous and fault tolerant — it is the edge of the system. A decision/calculation layer It must be pure and deterministic — that is where the result is computed, without randomness. The model is called inside One step to do one thing; he never decides which one will be next.
In practice, this means that The LLM answers questions — it does not decide which questions to ask next.
It's the difference between kitchen from a restaurant and the lounge. The kitchen is hot, dangerous and does not change every week — it is where hard work happens. The salon is the part that the customer sees and that you redecorate frequently. You don't put the stove in the middle of the tables, and you don't give the customer the key to the stock.
The problem: cognitive systems have two opposite load profiles. The "brain" (cognitive motor) is heavy, asynchronous, sensitive (keys, data, proprietary logic) and rarely changes. The "Vitrine" (interface) is light, synchronous, exposed to the public internet and changes all the time. Attending them in the same application is a classic trap: a failure in the exposed window compromises the engine, and each interface change risks logic.
The solution is to separate them physically and connect them by a narrow contract — typically an API with very few endpoints — never by a shared database. The brain processes in the background and public a snapshot ready; a display only read the result and collect feedback, returned in a controlled manner.
In practice, this means that the itera team in the interface every day without ever risking the sensitive motor.
Before you start the expensive engine, make the most of what is already structured — like a good cook who prepares everything you can to prepare before you light the stove. The golden rule of reliability is counterintuitive: use AI as little as possible.
The problem: each responsibility given to the LLM is a source of hallucination and a cost. Sending the model to "discover" something that a bank consultation, a business rule or an exact search already answers is to waste money and introduce risk where it did not need to exist.
The solution: map each field of the task and ask — "Can this come from a structured source, a rule, or an exact search?" If so, it's not AI's job. What remains — unambiguous free text, inferring intention, classifying tone — is the interpretative residue. He alone justifies the cost and risk of a model call. And this residue shrinks to each version, as you discover what else can be deterministic.
In practice, this means that Less AI on the way is at the same time less cost and more trust.
You never blindly trusted what the browser sends to the backend — always valid on the server. A strict schema is the same form validator, now applied to the LLM output: it is unreliable input by definitionUntil I pass the doorman.
The problem: if the template invents an attribute, hallucinates a format or returns score = 1.7 where the interval is 0–1, and this enters the system without validation, the error spreads silently — and only appears three steps later, unrecognizable.
The solution It's shielding every border with strict contracts. Good concrete practices: prohibit the unknown (reject any field outside the scheme instead of ignoring it); put the limits on schema, not on a if lost (it is the scheme that refuses 1.7); validates types statically when the language allows; and degrade item to item — in a lot, a malformed item is marked as uncertain, does not drop the whole lot. It also includes sanitizing adverse input before the Brain.
In practice, this means that validation fails immediately and cheaply, instead of spreading the error.
Imagine registering a whole new client with no bank history. The classic software operates under "closed world": what is not in the bank is fake. Intelligence systems need the opposite — the Open World: the absence of evidence is uncertainty, not denial.
The problem: "we found no sign that the client is dissatisfied" no It's the same as "the customer is satisfied" -- it's "unknown." Treating the gap as a "false" invents a conclusion where there is no evidence: the worst category of failure, a wrong business decision made with false confidence.
The solution It's got two faces. First, isolate the epistemic layers: evidence observed Second, make uncertainty a first-class state: what was not observed is valuable information ("cannot confirm X"), and sensor failures degrade to uncertainty, never to an invented value.
In practice, this means that teaching the system to say "I don't know" makes it more reliable and safer.
You already master pagetion and job queues to process large volumes without overthrowing the bank, and cache Not to repeat expensive work. AI FinOps is to apply these same standards to a resource whose "miss" costs a model call, not a SELECT.
The problem: treat token limits and credit with try/except on top of the limit rate error is fragile and amateur. A free flow turns the budget into a random variable — the same request can cost pennies or dozens of reais.
The solution: model the budget as persistent domain status — one ledger who survives between executions, knows how much he has spent, resets in the period and refuses spending as you approach the ceiling, before the provider refuses for you. Combine with three habits: early pruning (the cheapest filter first — garbage never reaches the LLM); resumable wave processing (each wave saves progress and to clean; the next pick-up from where it left off); and a cumulative corpus with idempotent key — if you have already been processed, do not sue again.
In practice, this means that The system gets cheaper per item as it grows.
How to test a system whose central component is stochastic — which can respond differently to each execution? The answer is isolating the stochasticity in the tests: you don't test if the model is "intelligent"; you test if the system around him remains correct when the model, network or data fail.
The problem: without objective criteria of "ready", "functioned" is a perception, not a verification — and elegant hallucinations go unnoticed. Even worse by delegating to autonomous code agents, who have no common sense to fill gaps: ambiguity given to an agent becomes a hallucination of scope.
The solution: Write scenarios in Gherkin (Data/When/So) before code with external calls replaced by fixtures recorded — zero API deterministic and cost test. Every scenario covers three axes: Happy path, degraded mode (an API drops, breaks limit or gives timeout — the system degrades gracefully) and open-world (an expected signal is missing). Reinforce with gates: a DoR prevents a task of entering open decisions; DoD Yeah. fail-closed — if it was not possible to conclude, the blockade is recorded and stopped at a safe place.
In practice, this means that an honest "LOKE" is the success of the process; a false "completed" is debt that wins with interest.
No one makes an important decision because "the algorithm gave 0,674". Compare with: "priorized because it signaled recent expansion and has a history of purchase in the sector; it was not possible to confirm the budget". The number is useless and does not inspire confidence; the justification, yes.
The problem: the "black box" is the largest corporate adoption blocker of AI. An opaque score generates user resistance, flooded with "why did the system suggest this?" in support and failure in explanatory audits.
The solution is to exchange the number for an actionable explanation with three elements: drivers in natural language (which weighed for and against), absent signs (Open-World honesty — showing the gap increases confidence more than hiding it) and source (from which source came each signal, for the user to check). Two cares make XAI reliable: manages the explanation by deterministic rules on the same object that generated the score — never with a second call from LLM, which would rationalize; and consider hiding the number and showing only the "why now" and "what remains to be known".
In practice, this means that an explicable and modest response wins an exact and opaque response in adoption.
Everything comes together in a real case: SocialSelling, a B2B lead prioritization system where the ten principles became product. The pipeline is deterministic in five modules — M1 Search, M2 Extraction, M3 Score, M4 Ranking, M5 XAI —, and the LLM acts only in M2 and M5; M3 and M4 are pure Python.
As the principles appear: the LLM is contained component (P1/P2) in a deterministic flow (P4); the local brain publishes snapshots and the portal never touches the engine bank (P3); strict schemas shield all border (P5) and ledgers govern requests and credits (P8); the operator sees "why now" and "what is missing", never score (P10). The absence of data becomes explicit uncertainty (P6/P7), and degraded BDD scenarios support fail-closed governance (P9).
The final turn: when the AI goes from copilot to autonomous developer, responsibility migrates to the machine — and this requires structure. The context exits the head and enters the repository (ADRs, SDDs, DoR/DoD versioned), because undocumented context is context that does not exist.
In practice, this means that the central discipline is to distinguish the implemented from the specified — know when no build.
Eight real scenarios — four over the ten principles, four over the five pillars. Do not decorate the rule; choose what you would do in production. Each answer explains why.
Hits: 0/8
You're building a legal contract analysis system. The system needs to: receive the PDF, extract clauses, classify the risk and generate a report.
1. Which architecture is correct?
A clinical screening system evaluates patients. For a newly registered patient, there are no tests for the last 12 months in the system — the data simply does not exist.
2. What does the well-designed system do?
A content moderation system assigned score 0.847 to a post. The human reviewer asks, "Why was this post signaled?"
3. What answer creates more confidence?
A moderation agent is processing 200,000 posts. At 3:00, it reaches the daily limit of tokens defined in the ledger. 60,000 posts are still missing.
4. What should the system do?
A support agent needs to consult requests and issue limited refunds. You'll connect it to the payment system.
5. What drawing respects the smallest privilege?
Its system validates every exit of the LLM against a strict schema, but the LLM itself still decides the order of the steps and what tools to call.
6. What stage of maturity is he in?
In a weekend hackathon you want to validate if a product idea makes sense. The prototype will be discarded on Monday.
7. What's the correct engineering here?
The consultations that a free agent discovered during an exploration became a report that the sales team began to consume every Monday.
8. What does the transitional rule now require?
Laboratory
Adjust engineering decisions and see impact — in real time and by a model deterministic (without AI): same controls, same results.
Basic values: Cost part 30 and Risk part 40 — reflect the overhead minimum of any production system (infrastructure, latency, network failures). Zero never It's possible in the real world.
What's going on in this architecture
Adjust autonomy or mark guardrails to see the impact of each decision.
Tip: high autonomy without schemas/BDD triggers risk and cost; ledger + cache drop cost; evals and observability increase auditability.
How much rigour does a task deserve? Place it by the two axes of the Spectrum of Autonomy — cost of error and reversibility — and see the quadrant and the recommended ceremony. The calculation is deterministic (without AI).
An exit may be perfectly valid and yet wrong. Contract and evaluation attack different faults.
Failed structural: wrong type, extra field, score = 1.7, JSON malformed.
Barred deterministically by contracts (P5), in runtime. Binary and cheap.
Output valid But semantically wrong: driftWell-formatted logical hallucination.
The contract Don't you see. Detected by Semantic evals / LLM-as-a-judge.
O LLM-as-a-judge wheel offline, out of the critical way — scores samples against a fixed item and aggregate metric (rate of drift), never decided on Runtime. Keep the IA zero on the critical track. The lesson: "You passed the contract."
The same deterministic pipeline, under different conditions. Pick a scenario.
In production, things fail. See how the same deterministic pipeline behaves in three different conditions — and why design It matters more than the absence of mistakes.
This 5-step pattern is repeated in different domains: prioritization of leads B2B, screening support tickets, content moderation, contract analysis. The structure is the same; the domain changes.
Laboratory
Add components and see the system evaluate architecture by deterministic rules — pointing out risks and good practices.
Click on the above components to mount your architecture.
To-Be Repository
When the AI goes from copilot to autonomous developer, responsibility migrates to the machine — and responsibility requires structure.
An agent has no intuition about past decisions. Undocumented context is context that does not exist — documentation becomes first-class infrastructure.
Code aesthetically perfect, but logically wrong. The trust that the code Beautiful inspires is what makes him treacherous. Vibe Coding without strong tests is not Agile — Unwise.
Software Design Document
The Source of Truth About What? The system is and its limits.
Architecture Decision Records
The immutable memory that explains the Why of past decisions.
Behavior-Driven Development
The Gherkin test contract — the agent's executable target.
Definition of Ready / Done
Task barriers: protect the agent from assumptions and validate the result.
Model Context Protocol
The agent's controlled hands and eyes to access tools.
Folder structure is a spatial concept — explore it as such. Click on a folder to expand; click on any item to see its function.
Click a file or folder to see its function.
Golden Rule — Versify Cognitive Status
O PROGRESS.mdthe agent's skills and configs enter the Git along with the
Code. So the rollback recovers AI code and reasoning at the same
time. Code and cognition evolve — and are restored — together.
Gates are also financial
A cost-free agent can loop in and become an unexpected invoice. The Gates need Iteration limit, budget and "fail and stop" when ceiling is hit — stop at a safe spot, as in front of a red test.
One Master Context long describes vision, scope and guardrails; the LLM generates the tree And the base files. Quick — but the foundation generated turns the agent's truth, then errors here spread. Invest more time in meta-prompt than in code.
In order: context folders → SDD base → 1st ADR → validation rules → 1st BDD → DoR/DoD. Only then the code. Slower, full control.
The two converge to the same logic: context and gates first, code later.
Quick consultation
The ten principles in a sentence, a glossary of the key terms and the case study that originated This content.
This content was extracted from a real B2B lead prioritization system — where the ten principles have become product.
Project's central discipline: distinguishing the implemented (PoC) specified (V1+) — knowing when no to build the complete apparatus.
SocialSelling is the case that originated this content, but the same principles apply to any system that uses LLMs: a technical support agent, a content moderator, a legal contract analyser or a clinical screening system. What changes is the domain — architecture remains.
Calculated metrics in your browser (localStorage) — nothing is sent to the server.
In short
AI Agent Engineering it is the discipline of building reliable agentic systems treating the output of the model as a unreliable input governed by architecture. The central rule: a reliable AI system has Very little AI on the critical path — rules and data solve the deterministic and the model only acts on the interpretive residue.
It's the pillar. M2 (Architecture) ed Trust Engineering.
Frequently Asked Questions
It is the discipline of designing reliable agentic systems treating each model output as unreliable input, validated by architecture (schemes, thresholds, borders). The objective is to keep little AI in the critical path: rules and data solve the deterministic and LLM acts only on the interpretive residue.
Solve with structured rules and data everything possible and reserve the AI only for the fraction of the task that requires the model. Same entry must produce the same output; non-determinism is isolated and testable.
The cognitive motor (heavy, with access to sensitive data) and the exposed interface are separate systems, connected by a narrow contract. The display never touches the bank directly; the brain never blindly trusts what comes from outside — reduces the surface of attack and failure.
Every inference carries a degree of confidence; the architecture defines a floor below which the output is intercepted before turning fact and degraded to a structured fallback (human revision, explicit uncertainty or deterministic route). If it was not possible to conclude safely, the system stops at a safe place instead of promoting a hunch.
The MCP is the security border by which the agent wins "mons and eyes" in a controlled manner, explicitly declaring which tools he can use. Each tool is typed by a schema with pre and post-conditions, which makes the actions of the agent auditable.
With a quota ledger: a persistent domain state that governs credits and spent and refuses expense before even calling the provider. Combined with early pruning (cheap filter first), it prevents the cost of calls to models from escaping control.