From Chaos of AI-as-magic to Discipline of AI-as-engineering.
Capacity comes from the model; trust comes from engineering.
CAOS · AI as MagicGOVERNANCE · AI as Engineering
M0 · Awakeningrecognizing the problemM1 · Mapreveal the truthM2 · Architectinggovern behaviorM3 · Orchestratescale intention
The arch of the journey goes from Chaos (IA as magic) to Governance (IA as engineering), passing
by four modules: Awakening, Maping, Architecting and Orchestrating.
A Trust Engineering is the practical implementation of the
Intentional Systems Model (ISM) v1.0 — a reference model which deals with
each AI output as an artifact governed by architecture, not by the hope that the
Prompt hit. It is a four-module journey that takes from AI-as-magic ed
AI-as-engineering. The central premise: the capacity comes from the model;
trust comes from engineering — and a reliable AI system has very little AI in
critical path.
M0 · Awakening — recognize
Silent Crash: AI failure that gives no error of
Code, but it loses business value.
M1 · Map — freeze the system truth (As-Is) in
Behavior Inventory auditable.
M3 · Orchestrate — scale the intention with
Spec-Driven Development and governed rendering
(A2UI), becoming a Cognitive Orchestrator.
Reading ~25 min · 4 modules · glossary of 21 terms · real cases (Zillow Offers, Epic
Sepsis, Mata v. Avianca) · aligned with NIST AI RMF and
ISO/IEC 42001. See also
Frequently asked questions.
You come in here asking, "Why does my AI fail?"Module 0 · Leadership Awakening
Central concept
The Silent Crash
The failure that doesn't light any red lights. AI does not fail as the classic software: there is no
stack trace, there is no exception — there is a lost business rule, an invented number
with confidence, a wrong decision made as if it were right. The failure is that
intentionNot code.
Architecture Note — Intensive Systems Model (ISM) v1.0. This journey,
Module 0 to Module 3, is not a collection of individual good practices: it is the
practical implementation of
— a reference model whose objective is to quality and deterministic clarity of
AI outputs. The ISM treats each output of the model as an artifact that needs to be
governed by architecture, not by the hope that the prompt will hit. The four modules are
the four phases of this implementation: recognize the failure of intention (M0), freeze the truth of the
the system (M1), impose execution guarantees (M2) and reverse orchestration intention
Everything (M3). Read what follows as the infrastructure specification of an intentional system.
The AI is an amplifier. Give it a clear organisation and it scales clarity;
deliver an organization with confusion and it scale the confusion — only faster and with
the appearance of competence. That's why the same tool produces miracle in a team and
disaster in another.
The danger is not the noisy error; it is the
.
The code compiles, the demo enchants, the result looks flawless — and yet it is
logically wrong or injures a rule that no one has documented. LLMs are extraordinary in
produce what Looks like it. perfect: clean syntax, well-chosen names, structure
convincing. That is exactly the trap — the appearance of correction without the substance.
We call it
,
And the trust that beautiful code inspires is what makes it treacherous.
Building AI on an organization that doesn't know what it knows is to build on moving ground.
Before speeding up, it takes a
:
knowing what is observed fact, what is inference and what is only supposition. Without that, the
AI speed only leads to the cliff faster.
This is the awakening of leadership: stop asking "Does AI work?" and start asking
"Is the AI doing the right thing, for the right reason, so I can audit?" A
next journey — mapping the truth, building trust, orchestrating intent — is the
engineering answer to that leadership question.
Classic failure × Silent CrashText equivalent
A task delivered to the AI follows two paths. If the code breaks, there is visible error —
noisy, but sure to correct. If it does not break, the demo delights; then the question
decisive is whether the intention has been fulfilled. When it's not gone and nobody notices, Crash comes along
Silent: loss of value without any warning.
Practice · Visible Failure or Silent Crash?
Rate each scenario. The absence of error is not absence of failure.
A report generated by AI sums up correctly, formats beautifully — but uses the table of
Last year's prices. Nobody notices for weeks.
An agent calls an API that no longer exists; the application launches an exception and deploy
IC pipeline failure.
A legal assistant summarizes a contract and leaves the termination clause unannounced —
Because she was in an attachment he didn't read.
💡
Business Benefit
The worst category of AI failure is not the one that breaks down the system — it's the business decision
wrongly taken with false confidence. Recognizing Silent Crash is the first step towards
make it impossible by architectureNot luck.
Real case · The cost of Silent Crash
When trust is high and reality has changed
The Silent Crash is not an academic hypothesis — it has already cost hundreds of millions. O
case below shows a technically "working" model while destroying value.
Real Estate / Financial
Zillow Offers — $500M+ loss
Situation
A neural network (Neural Zestimate) priced real estate for purchase
mass institutional, trained in data from a high liquidity market.
Silent Crash
After the post-pandemic macroeconomic turn, the price ratio changed. The model
followed suggesting inflation prices with Highest confidence — without
pop no alert (concept drift).
ISM Diagnosis
Not known:
and observability linked to business KPIs. The company operated at Magic:
request speed above an adjusted behavior inventory.
What the ISM would have changed
Monitoring the population deviation (PSI) and degrade to
fallback It would be safe to cross the threshold to interrupt the acquisition of thousands of
Overvalued real estate.
For an "intentional system," each phase needs to be auditable. In M0, two indicators
turn the ethereal failure of AI into explicit technical debt for executives.
Module 0 Success Indicators (Awake).
Metric
Formula / Collection
Orientation target
Silent Crash Index (CSI)
Executions with 200 OK technical but harmful impact on the total business of executions (retroactive collection).
< 5%
Coverage of the Evidence Matrix
% of business decisions with classified evidence (fact × inference × assumption).
> 80%
O CSI is the backbone of M0's narrative: it converts "the robot bought
expensive without making a mistake" in a number that the financial understands and prioritizes.
Sociotechnical dimension
The Human Journey of the ISM
The transition from "IA as magic" to "IA as engineering" is, first of all, a change of
mindset. Ignoring the human dimension generates resistance even when the technical argument is
Irrefutable. The ISM is anchored in Psychological Safety (Amy Edmondson):
without a safe climate to report anomalies, engineers camouflage degenerate models — and the
Small drift turns to "Normalization of Detour" (Diane Vaughan)
sank the Target Canada,
where analysts, fearing reprisals, disabled the alert of stock rupture while the
executive panel followed green.
Executive Leadership
Typical challengeResistance to "more process" when demo already works.
ISM StrategyTranslate Silent Crash into auditable financial risk; use the Evidence Matrix as an instrument of executive governance.
Technical Times
Typical challengePerception that reliable engineering is bureaucracy that brakes deliveries.
ISM StrategyDemonstrate with Devin+Salesforce case that SDD Hurry up. — eliminates rework caused by failure to intend.
Business Stakeholders
Typical challengeDifficulty in articulating the value of something that hasn't gone wrong yet.
ISM StrategyQuantify with retrospective CSI: identify silent crashes already occurring and their impact. The past justifies the future.
Checklist · Organizational readiness
Mark what's already true in your organization. Without JS, the checklist remains a guide
reading; with JS, he estimates his readiness to start the ISM.
" Point out a Silent Crash that you have already lived: what value was lost without that no warning
Would red appear?"
If you recognize one, you're out of Magic and into Consciousness.
Maturity 1 → 2
You already recognize Silent Crash. Now you need to reveal the truth of what exists.Module 1 · Mapping the Truth
Central concept
Reveal Implicit to Make It Explicit
Before modernising (To-Be), you need to freeze a high-fidelity photograph than the
system Really Do it today. Critical knowledge of business usually lives
hidden — on thousands of Apex lines, fragmented Flows and on the heads of few
experts. Modernizing without mapping is erasing vital rules without knowing.
Every legacy system suffers from "tribal knowledge": rules that sustain business but that
No one wrote it. The objective of Reverse Engineering Assisted by AI no Yeah.
generating eternal living documentation — is freeze As-Is with fidelity
sufficient for architects to decide, auditably, what to preserve, redesign or
Discontinue.
The strategy that defines success or failure is that of discovery. Read all by artifact type
generates isolated dictionaries without context; mapping by domain generates overlap. The approach
recommended is guided by critical journeys and risk (top-down with deep
dive: part of a business journey (e.g.: Lead-to-Cash), follows the trail of
implementation (call stack) and it maps every component it touches — isolating
quickly the dead code, which belongs to no journey.
In order not to fall into false positives, the AI crosses a
multifaceted: structural inputs (metadata, Apex, Flows), state (Custom Metadata,
Custom Settings, operational (Event Monitoring, Debug Logs — which reveal the "zombie code"
never executed) and humans (commits, tickets, interviews — giving the Why ed
rules exist). The final asset is not a catalog of IFs/ELSEs, but a
:
core rules, side effects and historical workarounds, each classified by
evidence trust.
Artefact chaining (M1 → M2). This inventory doesn't die like
Shelf documentation: it is the raw material of contracts. Each rule
core and each side effect catalogued becomes a clause in schemas for validation of
Module 2 — As-Is audited is what unambiguously defines what a valid output can and cannot
It can contain. Mapping the truth is not an archeology exercise; it is generating the specification
Executable of the following borders.
Discovery guided by critical journeys (As-Is)Text equivalent
The discovery starts from a critical business journey, follows the trigger by the call stack and
maps all the components it touches (classes, flows, integrations), producing the
Behavior Inventory (core rules, side effects, workarounds). The code that
no journey is identified as dead and isolated.
Practice · Fail-Closed Governance and Human Validation
The key point: the inventory no is accepted because the AI produced it.
Every behavior discovered goes through Human validation via PR Review and
by a posture
— when
the evidence is insufficient, the item is marked as hypothesis or Decision
open, never promoted to truth. An honest "unconfirmed" is worth
More than just a "confirmed" invented.
💡
Business Benefit
Mapping the truth before migrating avoids the worst risk of all modernization: finding out, in
production, that a silent rule worth millions was erased. The Inventory of
Behaviour transforms tribal knowledge in auditable asset.
Real case · Evidence mapping
When the evidence is contaminated
Mapping behaviors without isolating spurious correlations produces an inventory that confirms the
past rather than predict the future. This is what happened to a clinical model adopted in
large scale.
Health
Epic Sepsis Model — 67% of actual cases lost
Situation
Regression model integrated to medical records to predict sepsis, with very high
internal accuracy reported.
Silent Crash
Data leak (label leakage): medical interventions themselves
(e.g. antibiotics) were used to "prevent" sepsis — a confirmation
retrospective disguised as prediction. Result: alarm fatigue.
ISM Diagnosis
Failed to semantic mapping evidence. Without separating
evidence of inference, the inventory inherited a spurious correlation.
What the ISM would have changed
A
classify the relationship "antibiotic → sepsis" as a suspicious hypothesis, never promoted
As-Is truth without human validation.
The inventory is only reliable if it's measurable. These indicators expose how much As-Is was
in fact frozen — and how much is still hypothesis.
Module 1 Success Indicators (Mapar).
Metric
Formula / Collection
Orientation target
As-Is Coverage Rate
% critical journeys with full inventory and validated by human.
95% >
Pending Hypothesis Rate
% of behaviors not yet confirmed by human validation (honest Fail-Closed).
< 10%
Population Stability Index (PSI)
Distance between training and production distribution — early warning date/concept drift.
< 0.10 stable · alert on 0,25
Why PSI is the early alarm of the ISM
PSI compares histograms of the same variable between two periods. Market Convention:
< 0,10 stable; 0,10–0,25 minor deviation (attention);
> 0,25 significant change. When monitoring the orchestrator's output, the
team degrades predictive machine to the fallback Safe as soon as the PSI bursts
0.25 — exactly the brake that Zillow missed. Limitation: the PSI is univariate and loses
multivariate drifts; therefore it complements, and does not replace, the monitoring of
calibration.
"How much of the critical logic of your system lives only in the minds of a few people?"
If you already have an explicit and auditable inventory, you're at Maped level.
Maturity 2 → 3
The truth of the system is mapped. Now you build on it with rigor.Module 2 · Architecting Trust
Central concept
Trust comes from engineering
Capacity comes from the model; trust comes from engineering. A reliable AI system has
Very little AI on the critical path. Trust is not born from a smarter prompt —
born of more disciplined architecture.
After mapping the truth, it's time to build on it rigorously. The first principle
invariant is
:
solve with rules and data everything possible, and spend AI only on
— the fraction of the task that only the model solves. Same input must produce the same output;
non-determinism is isolated and testable.
The second principle is separation
.
The cognitive motor (heavy, sensitive, with data access) is a system; the exposed interface is
Another one. They communicate for a narrow contract. The window never touches the
bank directly; the brain never blindly trusts what comes from outside.
The third is the
across the border: LLM output is unreliable input until validated by a schema
strict that rejects unexpected fields. It's a firewall between the probability of the model and the
system determinism. On this basis, the isolation of epistemic layers is based.
(evidence)
(absence of data is explicit uncertainty, never "false"),
(FinOps) and weak-closed governance.
Finally, the
is the security and interoperability border: the standardized protocol by which the agent
wins "hands and eyes" in a controlled manner, explicitly declaring which tools can
use — a security border which, in Module 3, ceases to be data query and passes
governing until interface rendering ().
Artefact chaining (M1 → M2 → M3). Note where these schemas come from:
they are not invented in the vacuum — they derive directly from
Module 1. And once stable, they are no longer just defensive validators: with their
preconditions and post-conditions, they become the tool signature
(tools) that the Cognitive Orchestrator of Module 3 will consume via MCP. The rule
audited becomes a contract; the contract becomes a tool.
deterministic pipeline (DAG) — the key diagramText equivalent
Entry passes by deterministic validation and early pruning; only interpretive residue
reaches the LLM, which is invoked within a fixed stage. The model output is validated by a
strict contract requiring schema compliance and confidence of at least 98%: below
from this threshold or outside schema, it is intercepted and degraded to a structured fallback
(explicit uncertainty or human review); if valid, it becomes reliable output. A ledger of
quota governs the cost of the call. The LLM never decides what the next step is.
Brain × displayText equivalent
Vitrine (exposed, light interface) and Brain (heavy cognitive motor with access to
data) are separate systems linked by a narrow contract. The window never touches the
data directly.
Practice · "What if I no Do that?"
Each principle is an architectural decision, not a slogan. Open to see the consequence of
ignore him.
No hard contract on LLM exit
Risk: a field invented by the model (a price, a number, a status)
It crosses the system and reaches the customer as if it were true. No mistakes are made.
No quota ledger (FinOps)
Risk: API account bursts without warning. The cost only appears on the invoice —
the "cloud-surprise account" that kills AI projects.
No Open World (absence = uncertainty)
Risk: lack of data turns "false" silent. The system states with
trust which should admit not knowing — and the wrong decision is made.
No separation Brain × display
Risk: the exposed interface touches the sensitive seat directly. A Failure
in the window becomes a breach in the brain — the surface of attack and error explodes.
💡
Business Benefit
These practices attack the biggest blocker of corporate adoption of AI: distrust.
Deterministic-first cuts cost and error in the same move; contracts prevent data
invented to reach the customer; embracing uncertainty avoids wrong decision with false
trust. The result is a system in which executives and auditors feel safe
to trust.
Real case · The missing contract
Output of the direct Brain in the display
Without a strict contract at the border, the probabilistic exit of the model becomes fact without
reconciliation. The legal case below has become case law on the risks of this.
Legal
Mata v. Avianca — federal sanctions for fictitious citations
Situation
Lawyers used an LLM to substantiate a petition in the South District Court of
New York.
Silent Crash
Template returned quotes perfect in structure but nonexistent:
stocastically accurate to the legal standard, without adherence to reality.
ISM Diagnosis
Not known:
:
the exit of
passed straight to the window (the final petition), without reconciliation against base
deterministic.
What the ISM would have changed
Every statement generated would be validated by schema against a base of
real case law; a quote not found would shoot
.
O
It's only defensible if it's meditable. These indicators make the firewall between probability
and auditable determinism.
Module 2 Success Indicators (Architecture).
Metric
Formula / Collection
Orientation target
Threshold Interception Rate 98%
Calls to fallback Counts the "anomalies that didn't turn into an accident."
Monitor · system baseline
Excluded Calibration Error (ECE)
Mean difference between declared confidence and actual accuracy, by bin of probability.
→ 0 (well calibrated)
Lack of Invalid Contract Detection
Average time between invalid output and intercept by schema.
< 200ms
Statistical rigor: why ECE justifies the 98% threshold
Modern neural networks are miscalibrated — produce wrong predictions
with very high confidence (Guo et al., 2017). O softmax It's not a
Reliable probability. ECE measures this misalignment by partitioning predictions into
bins and considering the absolute difference between accuracy and trust in each.
A high ECE means that the "98%" declared by the model is not worth 98% — and legitimate to impose
an external brake (the Threshold) instead of relying on the model's own number. Limitation: o
ECE is sensitive to the binning and may mask rare but serious errors;
this is pair of PSI (M1), not substitute.
Market mapping
The ISM × existing frameworks and standards
The ISM does not replace its procedural legacy — it is the mortar that fills the grooves between the
High governance and engineering mats. Where it connects:
How the ISM integrates with established methodologies and global AI standards.
Framework / Norma
How ISM integrates
ISM entry
DevOps / CI-CD
The Behavior Inventory (M1) feeds the treadmill tests; rigid contracts (M2) saw gates Locking. The ISM changes the Fail-fast blind Fail-Closed.
M1 → M2
MLOps
The 98% threshold and the schemas complement the monitoring of drift; Fail-Closed is analogous to a circuit breaker models.
M2
Agile / Scrum
Spec-Driven Development (M3) is compatible with Definition of Done: each versioned spec is an executable acceptance criterion.
M3
DateOps
The Evidence Matrix (M1) formalizes the data line; the Open-World principle documents quality gaps rather than inferring on them.
M1
SRE
The Silent Crash is the failure of integrity that the HTTP 200 Hide. Extend SLIs to include fallback and rejections by schema.
M2
ITSM / ITIL
Fail-Closed and ledgers of quota have direct analogy with Change Management and capacity management.
M2
NIST AI RMF
Map adheres to M1; Measure/Manage M2; Government echo the M0. The ISM is the practical operational manual for the abstract controls of the RMF.
M0–M2
ISO/IEC 42001
The Inventory (M1) meets the mapping purpose; the Brain Taxonomy × Vitrine and Schema Contracts (M2) meet the technical controls of Annex A.
M1 → M2
The ISM in its context
Select the framework you already practice to see where the ISM fits — and the ISM trap
integration to avoid.
I come from DevOps / CI-CD
3 connection points
Rigid contracts have seen gates Pipeline blocking (dynamic ling).
The Behavior Inventory defines what the treadmill tests should cover.
Each deploy reverts code and the spec which originated it.
Trap: treat Fail-fast as absolute virtue. ISM imposes Fail-Closed — speed without brake is what produces the Silent Crash.
I come from MLOps
3 connection points
Monitoring of drift feeds the 98% threshold drive.
Fail-Closed is the circuit breaker the model at the time of the anomaly.
ECE and PSI become reliable SLIs, not just laboratory metrics.
Trap: evaluate success only by aggregated metrics (AUC, F1). ISM prioritizes unitary deterministic interception in edge.
I come from Agile / Scrum
3 connection points
Each versioned SDD spec is an executable acceptable criterion.
The ISM M3 becomes the Definition of Done Every task I call an LLM.
Epics and stories translate directly into specifications.
Trap: Sacrifice contracts to close Sprint. The technical debt of a Silent Crash costs more than the "perceived speed".
I come from DataOps
3 connection points
The Evidence Matrix formalizes data lineage and ontological origin.
Detected lacunas activate the Open-World Assumption, forcing abstention.
Metadata catalogue and the Behavior Inventory are reinforced.
Trap: let the inventory become dead documentation. He needs to spin on loop continuous against living specs.
↺ Feedback: lessons learned in M3 return as
Knowledge and feed the Arsenal Organization (Skills · Playbooks),
to refine the next mapping cycle.
ISM v1.0 artifact chaining. Three-layer data stream: na
Entry (M1), reverse engineering with human validation produces the Inventory
of Behaviors; their audited rules feed the Processing (M2)where
the architect derives the Validation Schemes (hard contracts). The pre- and post-conditions of these
schemas saw the signature of Output (M3) — the Tools displayed
via MCP / A2UI that the Cognitive Orchestrator consumes in the SDD stream. The lessons return as
Knowledge for the organizational arsenal, closing the cycle.
Behaviour is governed. Now the intention is scaled — without losing control.Module 3 · The New Maestria
Central concept
From Executor to Cognitive Orchestrator
Mastery is not in typing code faster — it is in directing machine intelligence
with clear intent. As an executive chef who does not cook every dish, but ensures that each
dish come out right, the professional rises from execution to the trial.
The mature Vibe Coding is not "prompts caos": it is structured intention and
persistent. The human provides clarity, context, examples and iteration; the machine
It materializes. When that intention lives in an ephemeral prompt, it gets lost. When you live in one
versioned specificationShe climbs.
This is why the central method is the
:
the specification is the anchor of intention. Instead of asking for "do X" and cheering,
The What? and Why — and the AI implements against This
spec, with the tests
() to check behavior. O
context comes out of the head and enters the repository, where the machine can read and follow.
The practical flow, reported by the author in the Devin + Salesforce case study, has four steps:
Spec (SDD definition of desired behavior) → Retrieve
(autonomous discovery of the relevant context) → Refactor (edited edition by
spec) → Validate (Appex Automatic Tests that check the criteria for
Accepted). Note how this closes the cycle of the journey: the spec records the As-Is/To-Be of Module 1,
and the tests make part of it verifiable under the contracts of Module 2.
Finally, the Organisational intelligence arsenal, which grows over time:
Skills (reusable procedures — risk-free refactoring, testing
exhaustive, documentation), Playbooks (process revenue — code review
iterative, onboarding, framework migration) and Knowledge (the asset that
accumulating — architecture and standards, conventions, lessons learned and drips).
Versing this cognitive state in Git is what allows you to reverse not only the code, but the
reasoning that produced it.
The thread sewing the journey (M1 → M2 → M3). Notice the chain
artifacts: o
(M1) generates the rules; the rules have seen schemas validation (M2); schemas, with
pre and post-conditions define the contract of tools exposed via MCP/A2UI; and
this set of typed tools that the Cognitive Orchestrator consumes in the SDD flow (M3).
The intention goes through the whole chain without getting lost — from what the system does today (As-Is) to what the
spec says tomorrow. This chain is the backbone of the
: each phase produces the
Input ruled from the next stage.
Spec-Driven Development FlowText equivalent
The SDD spec is the anchor of intention; from it the AI discovers the context (retrieve),
Spec-guided refactor and run the tests. If it fails, refactor; if it passes, the
Checked criteria were met — which does not prove the whole intention — and the lessons
They feed the organizational knowledge.
Practice · Map a task for the arsenal
Take a real task from your day to day and answer: each execution should leave the
Smarter organization.
Which Skill the execution (reusable procedure)? Which
Playbook the frame (process revenue)? Which
Knowledge does it feed (the learning that stays)? If the answer to
three for "no", the task ended and left no trace — intelligence evaporated with the
Prompt.
💡
Business Benefit
SDD and the arsenal turn AI from source of surprises into predictable collaborator. A
intentionality no longer depends on the individual talent of those who write the prompt and pass to
be a Organisation assets — versioned, audible and reusable. It's the
scale economy of intelligence.
Real case · Technical debt generated by AI
Code that looks ready and expensive after
Code assistants deliver apparent speed — and silent technical debt. A
orchestration without spec is what turns short-term gain into senior rework.
Technology / SaaS
GitClear analysis — jump in code churn and duplication
Situation
Analysis of over 150 million lines of code with strong adoption of assistants
encoding (Copilot style).
Silent Crash
Exponential growth of code churn (code redone in a few days) and
duplication — code syntactically great, but blind to the architectural context of
repository.
ISM Diagnosis
The AI dictated architecture instead of executing against a contract. The developer
became a passive reviewer of predictions.
What the ISM would have changed
Under
,
the specification (BDD/Gherkin) is the non-negotiable intention: the IA implements
against The spec, with tests as objective target.
Intention becomes active in the organization when it is measurable. These indicators show whether
the spec — and not the ephemeral prompt — is the reference of intention.
Module 3 success indicators (Orquestar).
Metric
Formula / Collection
Orientation target
Versioned Spec Coverage
% of features delivered with SDD spec in the repository.
> 90%
Arsenal Reuse Rate
Items of Skills/Playbooks triggered total of task executions.
"Your intention lives on a prompt that you throw away, or on a specification that your organization
Will you reverse and reverse?"
If the spec is the versed reference of your intention and knowledge accumulates, you are governed.
Maturity 4 → 5
Close of the journey
Where's your system/time today?
The rule "from Magic to Rule" is the axis of the journey — and mirrors consecrated models: the exit
empirical heroism ad-hoc for the measurable governance of CMMI and
progression Microsoft AI Adoption Maturity Model (from the demonstration to
orchestration governed). Click on a stage to see symptom, next step and KPIs
level — or do guided self-assessment.
Select a stage above to see the symptom and the next concrete step.
Discover my level — self-assessment in 5 questions
1. Do you recognize at least one Silent Crash who's ever lived?
2. Do you have an auditable inventory of what the system does today?
3. Does every border of AI go through a contract/scheme before it becomes a decision?
4. Does your intention live in a versioned spect (SDD), not an ephemeral prompt?
5. Does knowledge (Skills, Playbooks, Knowledge) accumulate and are reused?
This site practices what it teaches
Deterministic-first: content and navigation are static; nothing depends on calling an LLM in runtime.
Few AI on the critical path: the site does not perform live models — it is content cured and referenced.
Reuse on reinvention: the journey refers to the three source pages instead of duplicating them — you do not pay twice for the same processing.
Traceability: every technical concept points to the source file in the repository.
Intent versioned: the specification of this site lives in docs/specs/pages/engineering-trust/ — not in an ephemeral prompt.
"You don't just write code for the machine to run. You write the context that teaches
machine to be executed with your intention. Do this context well — and trust scale together."
Frequently Asked Questions
Frequently Asked Questions
Direct responses to the central concepts of the ISM v1.0 — for quick reading and for
response mechanisms (answer engines).
What is Trust Engineering (ISM v1.0)?
Trust Engineering is the practical implementation of
Intentional Systems Model (ISM) v1.0, a reference model that
treats each exit of AI as an artifact governed by architecture — not by hope
that the prompt hits. It is organized into four modules: Awake (M0), Mapear (M1),
Architect (M2) and Orchestrar (M3). The central thesis is that the capacity comes from the
model, but trust comes from engineering.
What is Silent Crash in AI systems?
O Silent Crash It's an AI glitch that doesn't light
no red light: no stack trace or exception, code compiles and demo
— and yet the result is logically wrong or injures a rule of
Undocumented business. It's a failure of intentionNot code. O
Zillow Offers case (damage of more than $500 million) is emblematic: the model followed
pricing real estate with very high confidence after the market changed without firing
No alert.
Why does a reliable AI system have little AI on the critical path?
Because trust is not born from a smarter prompt, but from architecture more
disciplined. In the approach deterministic-first,
structured rules and data solve everything possible and the AI is called only in
interpretative residue — the fraction of the task which
only the model solves. The smaller the probabilistic surface on the critical path, the more
auditable and reproducible is the system.
What is the 98% sure threshold and structured fallback?
O 98% Insurance threshold is the reliable floor below the
which all LLM inference is intercepted by architecture before becoming fact. When
the trust is below the threshold, the exit is not accepted as truth: it is degraded to
one structured fallback — ask for human review, answer "uncertain"
explicitly or trigger a deterministic route. He unifies
rigid contract,
Open World and fail-closed one
only operational criterion.
What is the first deterministic approach?
Deterministic-first means solving with rules and data everything possible and
reserve the AI only for the waste
interpretative. The same entry must produce the same output; non-determinism is
isolated and testable. It is the first invariant principle of Module 2 (Architect a
Trust).
What is Model Context Protocol (MCP) and A2UI?
O MCP (Model Context Protocol) is the security border and
interoperability by which an agent gains "hands and eyes" in a controlled manner,
explicitly stating which tools you can use. O
A2UI (Agent-to-User Interface) is the evolution of this pattern:
orchestrator uses MCP not only to query data, but for issuing intention to
interface — the Brain decides What? show and deterministic display
decide how render, extending the separation
Brain × display the presentation layer.
What is Spec-Driven Development (SDD) and why does it scale the intention?
No Spec-Driven Development the versioned specification is the
reference of intention: instead of asking "do X" and twisting, the What? and
Why, and the AI implements against this spec, with tests (BDD)
to check. Spec is not infallible: when spec, tests and code differ, a person
with authority decides what to fix.
When intent lives in an ephemeral prompt, it gets lost; when it lives in a specification
versed in Git, she climbs. It is the central method of Module 3, which transforms the Executor
ed Cognitive Orchestrator.
What are the four modules of the journey (M0 to M3)?
M0 · Awaken: recognize the Silent Crash and the failure of intent.
M1 · Map: freeze system truth (As-Is) in an Inventory of
Behaviour. M2 · Architect: impose determinism, strict contracts,
The 98% sure threshold and the MCP as a border. M3 · Orchestrate:
scale the intention with SDD, the arsenal (Skills, Playbooks, Knowledge) and the A2UI.
How does the ISM relate to NIST AI RMF and ISO/IEC 42001?
The ISM v1.0 is a pragmatic distillation of research lines and consolidated norms, not
an effort to disharmony the market. He's in line with NIST AI Risk Management
Framework (Map, Measure, Management, Government) and
ISO/IEC 42001:2023, and mirrors maturity models such as the
CMMI and Microsoft AI Adoption Maturity Model. Each
column is anchored in verifiable literature (see the
bibliographic basis).
How do I figure out the AI maturity level of my team or system?
The "Magic to the Ruler" rule has five stages: Magic (worked
in the demo), Consciousness (recognize Silent Crash),
Mapped (implicit became explicit), Architected
(contracts, determinism, MCP) and Ruled (Spec is the anchor of
Intention). The section Maturity brings a guided self-assessment of
five questions and an organizational readiness checklist to situate your team and
indicate the next concrete step.
Source documents
Reference guides
The three pillars supporting the ISM v1.0, published in Markdown for direct reading and download.
They are the source of the truth behind the cross references of each module.
🧭
Reverse Engineering Assisted by AI
The map from the past to the future: As-Is/To-Be, discovery guided by journeys, Matrix of
Evidence and the Behavior Inventory.
The ISM v1.0 is not an effort to disharmony with the market — it is a pragmatic distillation of
consolidated research lines. Each pillar is anchored in verifiable literature.
Academic foundations
Sculley, D. et al. (2015). Hidden Technical Debt in Machine Learning Systems. Neurips.
Anchors Silent Crash and Hidden Technical Debt (M0–M1).papers.nips.cc
Gal, Y. & Ghahramani, Z. (2016). Dropout as a Bayesian Approach: Representing Model Uncertainty in Deep Learning.
It supports predictive uncertainty and the Surety Threshold (M2).arXiv:1506.02142
Geifman, Y. & El-Yaniv, R. (2017). Selective Classification for Deep Neural Networks.
Math proof of abstention / Fail-Closed (M2).Semantic Scholar
Guo, C. et al. (2017). On Calibration of Modern Neural Networks.
It shows the "overconfidence" that justifies the ECE (M2).arXiv:1706.04599
Dekker, S. (2011). Drift into Failure.
Accidents such as incremental deviation in complex systems (M0).ResearchGate
Vaughan, D. (1996). The Challenger Launch Decision — "Normalization of Deviation".
Sociotechnical basis of Human Day (M0).Reference entry
Edmondson, A. (2018). The Fearless Organization — Psychological Security.
Condition to report anomalies without retaliation (M0).Harvard Business School
Martiire, C. (2019). Living Documentation + BDD (Dan North).
Living documentation and executable specification (M3).LLMs & BDD (arXiv 2024)
Reiter, R. (1987). A Theory of Diagnosis from First Principles.
Open-World Assumption formal root (M2).
Kleppmann, M. (2017). Designing Data-Intensive Applications, chapter 4.
Schema evolution and data contracts (M2).
Excluded Calibration Error (ECE). Visual explanation step by step.
Towards Data Science
Population Stability Index (PSI). Detection of drift date.
GeeksforGeeks
Access to knowledge: the source guides of ISM v1.0 are published in
Reference guides (Markdown format, direct access).
External links belong to the respective authors and editors, cited for educational purposes.
Engineering vocabulary
Unified Glossary
The four anchor terms that structure the narrative and vocabulary of engineering that the
sustains. Every word dotted in the text brings you here.
Anchore Terms
Silent Crash
Failed without code error, but with loss of value. M0 →
Interpretative Waste
The only part of the task where the AI is really needed — after the structured data solved everything else. M2 →
Mode Leak
When a disposable prototype becomes production without due rigor upgrade.
Intentional System
Where behavior is governed by architecture, not probability. M3 →
Engineering Terms
Syntactic hallucination
Code that Looks like it. perfect (syntax, names, structure) but is logically wrong.
Evidence Matrix
Multifaceted Pool (structural, state, operational, human) that prevents false-positive in discovery.
Behavior Inventory
As-Is holistic asset: core rules, side effects and workarounds, classified by trust.
Deterministic-first
Solve with rules and data; spend AI only on interpretive residue.
Brain × display
Physical separation between cognitive motor (sensitive) and interface (exposed), connected by narrow contract. Extended to the presentation, turns A2UI.
Rigid contract (scheme)
Strict scheme that rejects unexpected fields at LLM output; output is unreliable input until validating. Also rejects inferences below 98% Insurance threshold, forcing structured fallback.
Open-World / Open World
Absence of data is explicit uncertainty, never "false"; teaching the system to say "I don't know". Operationally, anchors the 98% threshold: below the confidence floor, the exit is treated as uncertainty, not as fact.
Quota Ledger
Persistent domain state that governs spent/credits and refuses expense before provider.
Early Pruning
Place the cheapest and discriminant filter as early as possible on the pipeline.
BDD
Data/When/Then scenarios with recorded fixtures; tests the system around the model, not the "intelligence" of the model.
Fail-closed
If it was not possible to complete, it is not marked as complete; the lock is recorded and stopped at a safe point. Fired when trust is below 98% threshold — instead of promoting a guess.
MCP (Model Context Protocol)
Standard protocol that gives the agent "hands and eyes" controlled; declares which tools you can use (security border). Evolve to A2UI when the agent uses it not only to query data, but to issue interface intention to be rendered deterministically.
SDD (Spec-Driven Development)
The specification versioned as an intention reference; the AI implements against it and the tests verify the chosen behaviors.
Cognitive Orchestrator
Who drives the machine's intelligence with intent (the "executive chef"), instead of executing each line. Consumes tools schema-derived types (M2) and can operate in mode A2UI.
ISM (Intentional Systems Model) v1.0
A reference model that treats each AI output as an artifact governed by architecture, not probability — focusing on quality and deterministic clarity. This journey (M0 → M3) is its practical implementation; each phase produces the governed input of the following.
98% Insurance threshold
Trust floor below which all LLM inference is intercepted by architecture and degraded to a structured fallback (explicit uncertainty, human review or deterministic route), rather than becoming a fact. Une rigid contract, Open-World and fail-closed in a single criterion.
A2UI (Agent-to-User Interface)
Architecture in which the orchestrator uses the MCP to emit interface intention: the Brain decides What? show (data structured under contract) and deterministic display decides how render — Separation Brain × display in the presentation layer.