Trust scores are convenient.
When there is a number, people feel reassured.
But there is a fatal misunderstanding about trust scores.
Trust is not an “estimate.”
Trust is a scope of permission.
Therefore, trust scores must not be treated merely as model outputs.
They must be treated as contracts.
This becomes decisive, especially in cold-start situations.
Why “Stages” Are Necessary
In a cold-start state:
-
There is no data
-
The distribution is unknown
-
The score is close to random
If trust is granted all at once under these conditions, the system will break.
What is needed instead is:
A structure that releases trust progressively
In other words:
-
Allow only low-risk actions
-
Expand permissions as behavior and evidence accumulate
-
Immediately degrade when uncertainty increases
If this is left to “learning outcomes,” responsibility disappears.
That is why it must be a contract.
What It Means to Treat Trust Scores as Contracts
Treating trust scores as contracts means:
-
Defining the meaning of the score
-
Fixing the scope of application
-
Making promotion conditions explicit
-
Making demotion conditions explicit
In short:
Do not let the score act as a proxy for judgment.
The score should be reduced to an input for boundary control.
The Model: Trust Ladder
The fundamental form of progressive trust is a ladder.
Example:
-
L0: Cold-start
-
L1: Observed
-
L2: Limited Trust
-
L3: Standard
-
L4: High Trust
The critical point is that each level is defined not by points, but by the set of permitted actions.
Trust Level = Contract of Allowed Actions
For example, in finance, issuance, or incentives:
-
L0: View only
-
L1: Low-value issuance only (with cap)
-
L2: Higher limits with frequency controls
-
L3: Normal operation (standard limits)
-
L4: Privileged (relaxed limits, simplified review)
Here, “trust” means:
How much impact the system allows the entity to have.
And this is a value judgment, not a learning result.
The Core of Contractization: Promotion, Retention, Demotion
To make the staged model contractual, at minimum the following must be fixed.
1) Promotion
-
What evidence triggers advancement
-
What counts as “proven behavior”
-
Over what observation window the judgment is made
2) Retention
-
What behavioral range maintains the level
-
How increased uncertainty is handled
3) Demotion
-
What triggers immediate downgrade
-
What triggers gradual downgrade
-
Whether audit logs and human approval are required
This is the part where judgment is externalized.
Not “Stage Score,” but “Stage + Score”
A common failure is:
Trying to decide everything with a 0–100 score.
The correct structure is:
-
Level (discrete): Defined by humans (contract)
-
Score (continuous): Estimated by AI
AI does not decide the level.
AI only produces evidence for level decisions.
Conceptual Contract Schema
A trust contract should minimally include:
-
trust_level: L0..L4 -
decision_mode: auto / recommend / human_required -
allowed_actions: explicit list -
limits: caps (amount / frequency / time) -
promotion_policy: promotion conditions (evidence / window / thresholds) -
demotion_policy: downgrade conditions (immediate / staged) -
evidence_requirements: required evidence -
audit: reference to evidence logs (EventsRef / SignalsRef)
The key point is:
Trust is implemented as a state machine.
Progressive Release Is About Evidence Sufficiency, Not Activity Volume
If trust elevation is based only on:
-
number of uses
-
cumulative amount
the system will fail.
Because it will Goodhart.
Promotion conditions must be based on a bundle of evidence, such as:
-
Quantity (count, volume)
-
Quality (anomaly rate, complaint rate, return rate, fraud signals)
-
Context (new device, location change, time-of-day shifts)
-
Diversity (detection of single-pattern concentration)
And this bundle itself must be contractualized.
When Uncertainty Rises, Trust Must Degrade
This is the critical property of progressive trust models.
Trust does not only increase.
It must fall when conditions change.
Examples:
-
Device changes
-
Behavioral shift
-
Out-of-distribution detection
-
Fraud signals triggered
In such cases, the system must support:
-
Immediate drop (e.g., L3 → L1)
-
or temporary freeze
This is the concrete implementation of fail-closed.
Place Human Approval at Stage Boundaries
Overusing Human-in-the-Loop kills operations.
Fully automating kills accountability.
Therefore, human approval should be placed only at stage boundaries, such as:
-
L0 → L1
-
L1 → L2
-
L2 → L3
In other words:
Humans own the responsibility for boundaries.
This is precisely Human-as-Author.
Conclusion
The contractual model of progressive trust scores means:
-
Treat trust not as a number, but as a permission boundary
-
Fix discrete stages by contract
-
Reduce continuous scores to evidence
-
Design promotion, retention, and demotion as a state machine
AI approximates the continuous.
But trust is discontinuous.
Therefore, trust must be protected by contracts.
コメント