AI is often believed to be fair.
It evaluates through numbers, remains unaffected by emotion, and produces consistent judgments.
But this assumption contains one critical oversight.
An objective function is written from someone’s values.
And in most cases,
those values are never made explicit.
A Score Is Not a Fact
When people see a score, they feel reassured.
0.82
75 points
Top 12%
It appears to be an objective result.
But a score is nothing more than
a compressed summary of
what was considered good
and what was ignored.
A score is not a fact.
It is a compressed expression of value judgments.
Where Did the Objective Function Cut the World?
When designing an objective function, choices are always made.
What to include.
What to prioritize.
What to leave out.
Yet in practice,
what is left out is rarely discussed.
Because it is:
hard to articulate,
difficult to reach agreement on,
likely to create friction.
As a result,
unwritten values are quietly discarded.
Implicit Judgments Have the Strongest Impact
There is something that influences outcomes more strongly
than the explicit weights inside an objective function.
It is
what was never considered in the first place.
Unquantifiable discomfort.
Long-term consequences.
Rare but catastrophic edge cases.
These do not appear in the score.
Yet they undeniably exist in reality.
AI is not “overlooking” them.
It is simply treating them as nonexistent.
When a “Fair Score” Becomes Unfair
A common misunderstanding persists:
“If the same objective function is applied to everyone, it is fair.”
But the real questions are:
Fair for whom?
Fair at whose expense?
An objective function may be:
rational from one value system,
violent from another.
And yet the score appears
with a neutral face.
Where Do the Values Humans Did Not Write Go?
This is the most important question.
The values humans intentionally or unconsciously
did not encode into the objective function —
Do they disappear?
Are they merely ignored?
The answer is neither.
They return from outside the system,
as problems.
Operational dissatisfaction.
Unexpected failures.
An indefinable sense of wrongness.
The better the score appears,
the deeper this misalignment can become.
AI Conceals Value Judgments
AI is not dangerous because it makes value judgments.
It is dangerous because it makes them invisible.
It compresses them into numbers.
It embeds them inside models.
It presents them as “results.”
In this process,
who chose which values disappears.
What remains is an unquestionable authority:
“The system says so.”
The Only Honest Design Attitude
There is no way to make an objective function “correct.”
But there is a way to treat it honestly.
That is:
to assume from the beginning
that there are values we did not write.
Concretely:
Explicitly state that the objective function is a hypothesis.
List what it does not capture.
Acknowledge the existence of judgments that cannot be quantified.
Do not delegate everything to AI.
Remain conscious of what was not delegated.
Conclusion
An objective function is a collection of values.
A score is merely a compressed judgment.
The values left unwritten become future problems.
AI makes value judgments invisible.
The human role is to take responsibility for what remains unseen.
AI is neither cruel nor fair.
It is simply executing, silently,
the values humans have written into it.
Concrete technical approaches to objective functions in machine learning are discussed in works such as Mathematics in Machine Learning and General Machine Learning and Data Analysis.
Those who are interested are encouraged to refer to those resources as well.
コメント