Article · Multidisciplinary foundation and validation programme
Formulation of problems like engineering aborted of uncertainty
The most consequent part of a decision is rarely to solve the problem. Is to find out what it is the problem, who has legitimacy to define it — and when to stop investigating.
Formular is the engineering of decision-oriented uncertainty reduction. And all the engineering of precise formulation of a stop rule: without it, refinement becomes cost, bias of Unwarranted confirmation and trust.
Figure 1 · The arc of refinement
Where do you come in?
The stopping criterion: marginal benefit against cost, delay and cognitive burden.
Go to Stop Rule → "Every area describes a different problem."Uncertainty is not one thing: it is six, and four of them no probability solves.
See the six uncertainties → " The more we study, the more confident we are — and not more certain."The paradox of overinformation and the penalty of hyperresolution.
See detachment → "How do I know this isn't new paperwork?"Eight criteria that, if observed, weaken the very hypothesis of this article.
See how to refute →- The thesis
- To form is the engineering of decision-oriented uncertainty reduction, and needs a stop rule. The article itself concludes that the hypothesis has partial support: the reduction of uncertainty is not monotonic, it is not universal and it is not enough alone.
- Format
- Article
- Depth
- In-depth reading · ~5 min for central idea · ~25 min complete
You prefer the essentials first? Read synthesis (~1 min)
The hypothesis Summary
What the thesis states — and what it does not say
In complex problems, the formulation guided by a structured bidding process, representation, validation and research produces more robust decisions than formulations ad hoc when reduces uncertainties relevant to the action, explains premises, preserves traceability, tests competing frameworks and achieves a condition of sufficiency operable before that cost, bias and complexity outweigh the expected benefit of the information.
The word that carries the argument is when. The hypothesis is conditional, and the article supports it only partially. That's why the whole page carries stamps: what is synthesis of literature, what is original proposal and what is not yet has been empirically validated appear separately, never shuffled.
Five things that the hypothesis does not affirm
- that any additional information improves the decision;
- that all uncertainty must be eliminated;
- that there is a true and unique formulation for the same problem;
- that all formulations must be quantitative;
- that the stop can be defined by a universal metric, valid in any domain.
The four subquestions
- How to represent states of knowledge throughout the formulation?
- How to measure uncertainties of different natures?
- How to assess the sufficiency of a formulation for a specific decision?
- How to detect the point where refining ceases to reduce relevant uncertainty and becomes a produce complexity, bias or unjustified trust?
This is the same principle that governs a reliable AI agent on another scale: stop at a Safe spot instead of promoting a hunch. View AI Agent Engineering.
Conceptal Delimitation Summary
Six uncertainties, not one
Reducing uncertainty to a probability distribution is convenient and misleading. What's Holdin' a real decision is almost never variance: it is not knowing what the goals mean, who It should be in the room, or what causal mechanism was left out of the model. The article decomposes the uncertainty in six dimensions — each with its defining question, its candidate metric and its limitation.
Figure 2 · Uncertainty dimension coverage
Dimension 1 Established
Random
What varies even when the model is correct?
- Candidate metric
- Entropy, variance, intervals.
- Limitation
- It requires an appropriate distribution — which already presupposes the model.
Dimension 2 Established
Epistemic
What do we not know about the phenomenon yet?
- Candidate metric
- Belief-plausibility, credal sets.
- Limitation
- Depends on the initial attribution of beliefs.
Dimension 3 Promising
Structural
What relationships, variables or causal mechanisms may be absent from the model?
- Candidate metric
- Model browsing; divergence between competing causal models.
- Limitation
- High computational cost — Monte Carlo nested simulations.
Dimension 4 Promising
Semantics
What do concepts and goals, as stated, really mean?
- Candidate metric
- Rate of divergent interpretations; inter-rater agreement.
- Limitation
- Agreement may reflect conformity, not convergence of understanding.
Dimension 5 Proposal
Border
Are the scope of the system and the stakeholders identified adequate?
- Candidate metric
- Default ex post of stakeholders, objectives or variables.
- Limitation
- It requires observation after the decision: it is only measured what it has already cost.
Dimension 6 Promising
Of values
What goals, preferences and criteria should govern the decision?
- Candidate metric
- Distance between rankings preferences (Kendall correlation).
- Limitation
- It does not solve genuine incommensurability between values.
- Classical Operational Research
- Proposed Framework
Long description of Figure 2. On a scale from zero to one, the classic Operational Research covers 0.95 of the random uncertainty and 0.72 of the epistemic, but only 0.30 of the structural, 0.15 of semantics, 0.10 of frontier and 0.10 of values. The proposed framework distributes coverage more uniformly: 0.80 random, 0.76 epistemic, 0.66 structural, 0.62 semantic, 0.72 frontier and 0.55 values.
Proposed Framework Original proposal
From disaggregated to decision: six states
The state of the formulation in an instant t is a tuple of twelve components — frames, objectives, values, alternatives, restrictions, evidence, hypotheses, models, uncertainties, traceability, legitimacy and provenance. An investigation transforms this state, and the processing is not monotonic: an interview can clarify objectives and, in the same movement, expose a conflict of values that was hidden.
Reading of the formula: the K state of knowledge in the instant t is the tuple of twelve components — frames, objectives, values, alternatives, restrictions, evidence, hypotheses, models, uncertainties, traceability, legitimacy and provenance. The State next, K in t plus one, is the result of applying the processing operator T to current state, research action a and observation y obtained in t.
An investigation action a produces an observation y and carries the state of knowledge K to the next state. The operator T is non-deterministic and nonmonotonic: can reduce uncertainty in one dimension and increase it in another.
Figure 3 · States of knowledge and the quality gate
- S0 Unbundled situation
- S1 competing frameworks
- S2 structured representation
- S3 validated model
- S4 restricted readiness
- S5 decision and learning
Minimum gate before deriving any solution (between S3 and S4): explicit framework · identified objectives · classified restrictions · recorded uncertainties · evidences cited · visible disagreements · explicit decision about what remained outside the scope.
State 0
S0 · Unbundled situation
Fragmented descriptions, inconsistent vocabulary, implicit goals, not stakeholders identified, causality uncertain.
- To get out of here
- Name the actors and separate what is observation from what is interpretation.
Status 1
S1 · Competing frameworks
Multiple frames explicit, identified border conflicts, objectives and values already separated analytically, documented divergences.
- To get out of here
- Register the framework divergence rather than dissolve it too soon.
Status 2
S2 · Structured representation
Entities and identified main relationships, operational objectives, alternatives enumerated, traced evidence.
- To get out of here
- Each statement carries epistemic status: fact, evidence, consensus, hypothesis, inference, opinion, value or premise.
Status 3
S3 · Validated model
Testable hypotheses, explicit causal models, consistency analysis performed, quantified or at least qualified uncertainty.
- To get out of here
- Passing through the quality gate — this is where the solutioneering He usually comes in uninvited.
Status 4
S4 · Restricted readiness
Robust decision in plausible scenarios, explicit residual risks, minimum legitimacy achieved, additional research cost already higher than expected benefit.
- To get out of here
- Declare the stop — and record the specific reason for stopping there.
Status 5
S5 · Decision and learning
Registered decision, preserved premises, outputs monitored, discrepancies Re-feeding the model.
- The Return
- From here comes the regression of the formulation: the world changes, the representation ages and the cycle begins in S0 or S1 — legitimate change and genuine degradation must be distinguished.
Combination
It merges new evidence to the previous base. When the result is a contradiction irreconcilable, this is a sign of explicit review — not silent disposal.
Focus (↓)
Extracts knowledge relevant to the decision-making field and discards the marginal. formalizes the closing of attention borders: always a choice, with cost, Never a neutral discovery.
The movement S0 → S2 is the same of making explicit a legacy system that no one has documented — see Engineering AI assisted reverse.
Video complement
From Theory to Practice: The Pipeline of the Computer Decision
So far uncertainty has been treated as an object of theory: six dimensions, six states, one Stop rule. Running it in a real system is another problem. It requires translating rigor conceptual for engineering — contracts, schemes, auditable decision points — without the translation dissolves precisely what made the formulation defensible.
Three points in the video talk directly to what's above. The first is the Trap of Solutionism: the same solutioneering that the gate between S3 and S4 exists for — when the preferred solution comes before the problem, the rest of the process becomes Justification. The second is the transformation of human narrative ed measurable formulation by PFQi (Formation Quality Index): passage from the ambiguous account to something that compares, versifies and audits. It is worth reading it with except for Panel, not single index — the index is summary diagnosis, never cut-off decision.
The third is the non-negotiable limit: human supervision at the border of machines. A pipeline reduces factual uncertainty, orders evidence and refuses to inference that does not reach the threshold; it does not decide what consequences count, nor assumes the responsibility for them. It is the principle of fail-closed ed AI Agent Engineering applied to whom responds by decision: when it cannot be concluded safely, the system stops and Returns the choice to a human.
Video · The Pipeline of the Computer Decision
The Stop Rule Not validated
When you stop investigating
The naive version of the rule — investigate as long as the expected value of the information exceeds the cost — ignores two things that decide the result in practice: cost of delay and the specific value of investigations that do not reduce uncertainty parametric, but they discover an entirely new framework.
Reading the formula: stop investigating when the sum of three expected gains from the next round — the value of the information sample (EVSI), decision-making utility gain (DUG) and the value of discovering a new framework — is lower than the sum of three costs: direct to the next investigation, the cost of delaying the decision and the cost of overload cognitive.
Stop when what the next investigation promises — information sample value (EVSI), gain in decision-making utility and chance to discover a New framework — is lower than what it charges: direct cost, cost of deciding more late and cognitive cost of additional complexity.
Figure 4 · Where the curves cross
- Marginal benefit (EVSI′ + value of new framework)
- Marginal cost (direct + delay)
- Total cost with cognitive load (λ)
- Breakpoint
- Breakpoint
- 4,0 rounds
- Cumulative net benefit
- 164teaching units
- Scheme
- Short Investigation — Stop Before Detail
Long description of Figure 4. Three series over ten rounds of investigation. O marginal benefit starts at 130 units in round zero and falls continuously: 55 on Round 2, 27 in round 4, 15 in round 6 and 8 in round 10. The direct marginal cost more Delay starts at 16 and climbs straight up to 36 in round 10. The total cost, adding to cognitive load governed by lambda, starts at 16 and reaches 53 in round 10, with accelerated growth. The curves cross in round 4, which is the stop point of the Standard scenario: before she investigates pays off, after she doesn't.
See standard scenario numbers (λ = 0.4 · delay cost = 2)
| Round | Benefit | Total cost | Liquid |
|---|---|---|---|
| 0 | 130,0 | 16,0 | 114,0 |
| 1 | 83,0 | 18,4 | 64,6 |
| 2 | 55,5 | 21,4 | 34,0 |
| 3 | 38,2 | 24,8 | 13,3 |
| 4 | 26,8 | 28,6 | -1,8 |
| 5 | 19,1 | 32,8 | -13,7 |
| 6 | 13,7 | 37,3 | -23,5 |
| 7 | 9,9 | 42,0 | -32,1 |
| 8 | 7,2 | 47,1 | -39,9 |
Decisional sufficient
Seven conditions: explicit critical objectives; key alternatives identifiable; relevant restrictions represented; treated high impact uncertainties; robust decision to plausible variations; relevant perspectives considered; and gain expected to investigate further no longer compensating for the cost.
Negotiated efficiency Proposal
When there is no purely mathematical stop rule: participants recognize their perspectives on representation, the main differences are explicit, the consequences are understandable and further investigation is no longer justified. It's not I would like to thank the rapporteur for his report. will be reviewed.
An honest restriction of the apparatus itself: EVSI It's only calculable when alternatives, states and utility function are already specified — i.e. from S2. Before that, the decision to investigate is based on qualitative heuristics, not in exact calculation.
Contrary evidence Evidence
The paradox of overinformation
The strong hypothesis that "more information improves decision" is rejected by evidence. In the classic study by Slovic (1974) with specialized prognosticators, the increase in the number of available variables quickly stabilised accuracy — while confidence subjective of the judges kept rising. Peng et al., with neurophysiological measurements in shopping environment online, found compatible effect: more information, more time decision, less attention per item and signs of regret.
Figure 5 · Accuracy saturated, confidence not
- Predictability
- Subjective trust
Reading of the formula: the lambda hyper-resolution penalty at instant t is the sum of four plots. First, a basal penalty lambda zero. Second, the coefficient lambda one multiplied by the highest value between zero and the ratio between confidence and variation of accuracy, subtracted from the maximum acceptable ratio rho — i.e. There is only penalty when trust grows proportionally faster than accuracy. Third, the lambda coefficient two multiplied by marginal complexity X. lambda coefficient three multiplied by bias indicators B.
When trust grows faster than accuracy above acceptable reason ρ, a penalty rises; X is the marginal complexity introduced and B are bias indicators detected. The usefulness of the next investigation becomes EVSI(a) − C(a) − λt · ΔX(a).
Formalization does not prove that trust and accuracy have a linear relationship, that trust can be be measured without bias, or that λ is transferable between domains. The article is explicit: λ must start as experimental variable and not be incorporated into high decision support systems risk before validation. It's the same discipline Silent Crashapplied to the decision-making process: The danger is not the noisy error, it is the trust without ballast.
Measurement Original proposal
Panel, not single index
It is tempting to reduce the quality of a formulation to a number. The article builds this issue — and then argue against it: a composite index hides exactly the trade-off that interests the decision maker. A formulation can score high in completeness and low in legitimacy, And the weighted sum erases that information. The defenseable alternative is the panel, with gate minimum by dimension.
| Size | Candidate metric | Maturity | Limitation |
|---|---|---|---|
| Random | Entropy, variance, intervals | established | Requires appropriate distribution |
| Epistemic | Belief-plausibility, credal sets | established | Depends on the initial attribution of beliefs |
| Structural | Model browsing, divergence between causal models | promising | High computational cost |
| Semantics | Rate of divergent interpretations; inter-rater agreement | promising | Agreement may reflect conformity, not truth |
| Border | Default ex post stakeholders, objectives or variables | proposal | It requires observation after the decision |
| Of values | Distance between rankings preferences | promising | It does not solve genuine incommensurability |
| Consistency | Logical violations between objectives and restrictions | established | Consistency does not imply validity |
| Traceability | Coverage of links between objectives, evidence and decisions | moderate | May encourage superficial documentation |
| Legitimacy | Participation, challengeability, recognition of prospects | qualitative | Hard to quantify without reducing the vote |
| Ecological validity | Accuracy of outputs and contextual stability | promising | Requires longitudinal data |
The minimum gate suggested does not aggregate: requires integral logical consistency and coverage of stakeholders above a threshold, leaving the other dimensions visible side by side. All metric here is subject to Goodhart's risk — target, stop measuring — and so the article requires adverse indicators with each.
State of art Summary
Seven traditions, seven borders
The gap is not absence of components: they exist, mature. It's absence of a layer that Turn them on. Each tradition operates in a section of the process and ends where the following assumes that the work has already been done — and no one covers the last stretch, that of learning after the decision.
Figure 7 · Where each tradition begins and ends
- Problem Structuring Methods
- Structure plural situations by dialogue (SSM, SODA, Strategic Choice, Value-Focused Thinking). Assessed by consensus and perceived learning, rarely by outcomes comparable decisions between cases.
- Decision Analysis
- Alternatives, states, consequences, expected utility, EVPI/EVSI/ENBS. Take as given the space of alternatives, the causal model and the list of objectives.
- Information Theory
- Entropy, mutual information, minimum description; information algebras give the formal apparatus to combine and focus knowledge. It does not measure the quality of the framework or detect the variable that no one thought to include.
- Bayesian Decision Theory
- Formular is to design an experiment; the expected information gain guides what to investigate. Measure parameter entropy reduction within an already specified model — uncertainty about the model itself remains the most expensive frontier.
- Cognitive Science
- Borders of attention (Simon), mental picture (Klein), information foraging (Pirolli), detachment between trust and accuracy (Kahneman, Slovic). Explains why Stopping is necessary — and warns that the modeler is never a neutral observer.
- Requirements and Systems Engineering
- Completeness, consistency, verifiability, traceability (ISO/IEC/IEEE 29148); processes mission, architecture and validation. Receives the problem already defined: document quality does not equals decision-making quality.
- Knowledge Engineering
- Ontologies and graphs make the formulation auditable, with an open world assumption — absence of affirmation is ignorance, not denial. Organizes meaning but does not ponder preferences No decision-making authority.
See the complete matrix of literature (12 works, contribution and limitation)
| Authors | Contribution | Limitation |
|---|---|---|
| Simon (1973) | Misstructured problems treated by closing cognitive boundaries | Oriented to formal puzzles, not socio-technical decisions |
| Checkland (2000); Eden & Ackermann (2001) | Participative structuring of plural situations | Deliberate refusal of mathematical formalization |
| Montibeller & Franco (2010) | Iteration, dialogue and risks of premature structuring | Single case study; no metric validation nor causal effect |
| Cinelli et al. (2020) | Taxonomy of the MCDA process | Taxonomy, not epistemic state theory |
| Rothery et al. (2020); Keisler et al. (2014) | Good practices of EVPI/EVPPI/EVSI/ENBS | Conditional to model specification, priors and alternatives |
| Montgomery et al. (2022) | Systematic mapping of requirements quality attributes | Quality of the requirement does not equal the decision-making quality |
| Russell & Wefald (1991) | Metaracinium: computing value and optimal stop | Depends on a priori calculable marginal utility |
| Gettinger et al. | Effect of representation on the decision-making process | Limited to visual representation |
| Peng et al. | Information overload reduces attention and raises regret | Transactional area; measures process, not normative |
| Slovic (1974) | Disengagement between accuracy and trust as information grows | Specific field of specialized prognosis |
| Hoppe (2018/2024) | Practical rules for structuring; questions the reification of wickedness | Still no comparable general metric |
| Browne & Pitts (2004) | Cognitive rules of stopping for information | Search online; not necessarily complex decisions |
Contrary evidence Summary
Where this does not apply
In social and planning problems, the formulation itself is the problem: there is no criterion univocal stop because the actors differ on what consequences count, no Just about facts. Reducing factual uncertainty does not solve a normative disagreement — and forcing the mathematicalization of immeasurable values can alienate the parts, giving appearance of objectivity to a choice that is normative. The defensible formulation needs to name the state "unsolved normative disagreement" without treating it as an engineering failure.
Five domains of inapplicability
Routine Problems
Well defined: formalizing generates bureaucracy without benefit.
Tactical crises
The time available is shorter than the time to model; specialized intuition tends to win.
Chaotic environments
The rules change faster than the formulation artifact can be updated.
Deep political conflicts
The central problem is power and recognition, not knowledge.
Extreme urgency
The cost of delay dominates any expected amount of additional information.
Four unneutralized philosophical criticisms
Reification
Calling it "engineering" suggests an object ready, separable from the actors and institutions that constitute it. Vocabulary pushes in this direction; using it requires awareness of it.
Goodhart Risk
Metric adopted as target ceases to measure: completeness becomes irrelevant items; consensus becomes silencing; trust becomes persuasion; stability becomes rigidity.
Incommensurability
Security against autonomy, efficiency against equity: divergences that may not allow legitimate numerical aggregation.
Viés in the Elicitation
The order of questions and the choice of who is interviewed first shape the result. The modeler is never a neutral observer.
Empirical program Not validated
How to refute this hypothesis
A framework that doesn't say how it can be dropped is not a research program — it's a preference. The central hypothesis will be considered weakened if, in controlled studies, pre-registered, any of the eight results below is observed.
Structured formulation only equals methods ad hoc in decision quality, after equalizing time and training.
The process increase cost and time without compensatory decision-making improvement.
Gain in subjective confidence does not come accompanied by gain in calibration or real performance.
The effects disappear under replication in multiple heterogeneous domains.
The intervention increase exclusion of stakeholders, premature closure of the frame or cognitive overload.
A minimum protocol, without the formal apparatus, produces statistically indistinguishable results.
Estimate ecological validity or structural uncertainty cost more resources than the risk you want to mitigate.
The formalisation reduces contestability or produces superficial acceptance without representing the affected groups.
Strong support would require the opposite: replicated effect, with relevant size and range of informed confidence, in at least three heterogeneous domains, against active and practical control — and with the planned mediator explaining part of the gain without the effect being reduced the longer contact with the problem or the more unjustified trust.
See the proposed benchmark (ten parameterized cases)
| Case | Domain | Complexity | Stakeholders | Urgency | Data |
|---|---|---|---|---|---|
| B1 | Server capacity allocation | Low | Low | High | High |
| B2 | Priorization of technical backlog | Medium | Medium | Medium | High |
| B3 | Choice of critical supplier | Medium | High | Medium | Medium |
| B4 | Infrastructure expansion | High | Medium | Low | Medium |
| B5 | Water management | High | High | Medium | Low |
| B6 | Urban mobility policy | High | High | Low | Medium |
| B7 | Cyber Incident Response | High | Medium | Too high | Medium |
| B8 | Adoption of AI in attendance | Medium | High | Medium | High |
| B9 | Prioritisation of social policies | Too high | Too high | Medium | Low |
| B10 | Climate adaptation | Too high | Too high | Low | Low |
Each case requires deliberately incomplete initial description, stakeholders simulated with conflicting objectives, hidden alternatives, competing causal models, ground truth Partial and time-delayed outcomes — without this, only ability to produce more complete documents.
Institutional implications
The right to formulate
Formling a problem is an act of power: it defines what will be told, who will be heard, which damages enter the account, what alternatives exist and what consequences remain invisible. Treating the formulation as a preliminary technical step conceals this dimension — and naturalizes asymmetries under appearance of methodological neutrality.
Liability
Register authors, stakeholders consulted, consciously excluded groups and justification for exclusion. In irreversible decision, the cost of delay must be compared to the cost of error — never assumed as dominant.
Transparency
It is not to reveal the mathematical apparatus: it is to allow those affected to understand how the problem was framed and how to contest. Auditable system without a challenge mechanism is not transparent — it is only documented.
Learning
The formulation is organizational memory. Without preserving state, rejected alternatives and reason for stopping, the organisation cannot distinguish execution error, model error, framework error or legitimate context change.
Conclusion
Verdict: partial support
The formulation of problems may be modeled as a process of reducing the uncertainties relevant to the decision. But this reduction is not monotonic, it is not universal and it is not It's enough on its own. It must be stopped before cost, bias and complexity overcome the expected informational benefit — and needs to accommodate situations where the central problem It is not factual, but normative.
The potential contribution is not in replacing Problem Structuring Methods, Decision Analysis or Requirements Engineering, but in offering the layer that represents as a formulation changes over time, what uncertainties it reduces and what creates, when the decision is makes it defensible and — the most neglected point — when the refinement itself passes to degrade the quality of the decision-making process.
What Synthesis Is Summary
The bridge between PSM and Decision Analysis; the use of EVSI to reason on additional research; quality attributes of Requirements Engineering; limits documented cognitive; the separation between parametric and structural uncertainty; ontologies and causal models as representation.
What is proposed unvalidated Not validated
The ontology of formulation states; border uncertainty as first class dimension; ecological validity and decisional sufficiency in the same vector; the formulation regression model; the penalty λ; the negotiated sufficiency; ten cases; and the experimental protocol.
Agenda, in order of precedence: to stabilize the minimum ontology; to operationalize coverage; traceability, ecological validity and legitimacy; test λ under controlled conditions; publish an initial benchmark, albeit reduced; running the first controlled study; monitoring cases real longitudinally; and compare the framework with PSM and practice ad hoc — instead of take your superiority a priori.
Vocabulary
Glossary
- Formulation
- The process of building a representation sufficiently explicit to support reasoning, comparison, investigation or action. It's transformation, not preliminary stage.
- Engineering aborted
- Engineering of the formulation deliberately subject to stopping rules, so that refinement continuous does not degenerate in cost, confirmation bias and false trust.
- Decisional sufficient
- Contextual property: explicit critical objectives, identifiable alternatives, represented restrictions, high impact uncertainties dealt with, robust decision, prospects considered and gain from investigating no longer compensating the cost.
- Negotiated efficiency
- Stop criterion for problems where there is no mathematical rule: interim agreement on what will be decided, what differences remain open and when there will be a review.
- EVSI
- Expected Value of Sample Information — expected value of a specific investigation and partial. Supposing alternatives, states and already specified utility; EVPI is the case perfect information limit, and ENBS takes the cost of its own investigation.
- Overresolution penalty (λ)
- Proposed penalty that grows when trust rises faster than accuracy, when the marginal complexity increases and when there are bias indicators. Experimental variable, no known constant.
- Belief-plausibility
- Pair of measures of evidence theory: a belief is the minimum support that evidence gives a chance and plausibility is the maximum still compatible with They. The interval between the two measures how much one ignores, rather than forcing a Single probability.
- Credit sets
- Set of probability distributions treated as equally admissible when The evidence is not enough to elect one. They represent epistemic uncertainty without pretending There's no such thing as precision.
- Model browsing
- Instead of choosing a single causal model, it considers the forecasts of competing models for each other's credibility. That's how you estimate uncertainty. structural — and the computational cost is the price of that honesty.
- Kendall Correlation (τ)
- Measure the agreement between two orderings counting the pairs that appear in the same Order in both. Here is to measure the distance between the rankings preferences of different stakeholders — uncertainty of values.
- Border uncertainty
- Uncertainty about the adequacy of the scope of the system and the identified stakeholders. Measure by omissions discovered after the decision.
- Ecological validity
- The degree to which the formulation corresponds to the relevant functioning of the real domain — distinct internal validity, predictive, construction and legitimacy.
- Solutioneering
- Allow a preselected solution to silently define the problem. The gate between S3 and S4 exist to make this movement visible.