What Are AI Defects? — The Core of AI Yield

In Manufacturing, There Is a Fundamental Question

In manufacturing, there is one fundamental question.

That question is:

What is a defective product?

In factories,

before producing anything,

the definition of defects

is determined.

Why?

Because without defining defects,

quality control is impossible.

---

AI Systems Still Lack This Question

Interestingly,

many AI systems

still do not have

this fundamental question.

---

AI Produces Decisions at Scale

AI systems

mass-produce decisions.

• Fraud detection
• Recommendations
• Credit scoring
• Ad delivery
• Customer support

All of these systems generate:

• Tens of thousands
• Millions
• Hundreds of millions

of decisions every day.

In other words,

AI is a factory of decisions.

And in every factory,

there is something that always exists.

That is:

defects.

---

What Are AI Defects?

Simply put,

AI defects are:

decisions that are not socially acceptable.

For example:

• False blocks
• Incorrect charges
• Wrong bans
• Misdiagnoses
• Irrelevant recommendations

However,

AI defects are more complex than they appear.

---

Three Types of AI Defects

AI defects can be broadly classified into three categories.

---

1. Wrong Decisions

This is the most obvious type.

The AI produces

an incorrect decision.

For example:

• Flagging legitimate users as fraudulent
• Rejecting safe transactions
• Providing incorrect answers

This is primarily

a model accuracy issue.

However,

most serious AI failures

do not occur here.

---

2. Illegal Decisions (Decisions That Should Not Be Made)

This is actually

the most dangerous type of defect.

The AI makes a decision

in a situation where it should not decide.

For example:

• Insufficient data
• Unknown patterns
• Low confidence
• Ethical considerations
• High-impact outcomes

In such cases,

the AI should

stop.

However, without proper boundaries,

the AI continues

to make decisions.

This is

a design defect.

---

3. Untraceable Decisions

There is another critical type of defect:

decisions that cannot be explained.

For example:

• Why was this customer rejected?
• Why was this ad shown?
• Why was this answer generated?

If there is no:

• Log
• Decision rationale
• Decision rules

then the decision becomes

unauditable.

As a social system,

this is

a defective output.

---

Structure of AI Defects

AI defects can be summarized as follows:

The key point is:

Most AI defects are not caused by the model.

---

The Essence of AI Yield

AI yield is defined as:

The proportion of decisions that are not defective.

In other words:

AI Yield = Total Decisions − Defective Decisions

---

How to Improve AI Yield

The important insight is:

Improving the model alone is not sufficient.

What truly matters is:

• Boundary design
• Decision design
• Logging architecture

---

Lessons from Semiconductor Manufacturing

This is similar to semiconductor factories.

Semiconductor companies did not improve yield

by making chips perfect.

They improved yield through:

• Inspection processes
• Redundancy design
• Failure analysis

AI will follow the same path.

---

The Essence of AI System Design

Building an AI system

is not about building a model.

Building an AI system

is about

designing defects.

---

In other words:

• What is considered defective?
• Where should the system stop?
• How should decisions be recorded?

These must be designed.

---

The Structure of AI

AI produces signals.

Decision determines actions.
Boundary stops defects.

And as a result,

AI yield is determined.

---

The Future of AI

The future of AI

is not about model competition.

The future of AI

is about

quality engineering.

Chinoba — Runtime Society and Coordination Systems:
chinoba.org