Scientific Thinking (3) Confirmation Bypass and Quadrant Tables

Scientific Thinking (3) Confirmation Bypass and Quadrant Tables

In the previous article, we discussed various inference methods for inferring hypotheses, especially hypothetico-deductive methods. In this article, we will discuss how to test the hypotheses we have deduced.

Lessons in Scientific Thinking

After the hypothetical deduction described in the previous article, it is necessary to verify whether or not the hypothesis is correct in order to prevent it from becoming a mere statement. The word “verification” is sometimes used to mean a detailed investigation, but its original meaning is the act of confirming whether or not a hypothesis is correct.

To think about verification, consider a case where you are given a sequence of three natural numbers, and you answer yes if the sequence fits the rule in your mind, and no if it doesn’t. Then consider a case where you give some hint to infer the answer. If you are given a hint that says yes to “2, 4, 6”, you can infer the answer to some extent if all the numbers fit the same rule (for example, they are even numbers), but if each of the three numbers has a different rule, you will not be able to reach the answer.

In such cases, it is not enough to be given only the numbers of the correct examples, but in order to find a better hypothesis, we must be presented with something that does not fit the hypothesis (counterexample), or we will never reach the correct answer. However, many people fall into the trap of only looking for examples that fit the hypothesis (thinking only about the hypothesis they are predicting) when they are sure that this is the case. This is called “confirmation bypass.

As an example of how to think about this confirmation bypass, there is the “four-card problem” (Wayson’s choice). Suppose you have four cards in front of you, each with a letter on one side and a number on the other. For example, if the cards are “A”, “K”, “4”, and “7”, the question is, “If one side is a vowel, the other side of the card must be an even number”. This is a problem.

In contrast, a person who is experiencing confirmation bypass will only check “A” and “4”, which satisfy the rule. This choice will confirm the correct one, but not the one that doesn’t match the rule, if any. In this case, the logically correct answers are “A” and “7”, and if the back of “7” is not a vowel, all the rules can be confirmed to be true. (In this rule, the back of a consonant can be anything, the back of “K” is allowed to be either an even radix, and the back of “4” is allowed to be either a vowel or a consonant.)

The ad hoc hypotheses I mentioned in the previous section are based on this confirmation bypass, which reduces the confidence level of the verification.

Thus, in order to test a hypothesis, counterexamples are important as well as positive examples, and the verification experiment to test the hypothesis needs to be based on these ideas. In this section, we will discuss the evaluation of the quadrant table used in machine learning. In general machine learning, such as classification, the evaluation of test data is done in four decomposed regions as shown below.

First of all, TP (True Positive) is called True Positive, which is judged as correct by machine learning; TN (True Negative) is called True Negative, which is judged correctly as not correct by learning results; FP (Flase FP (Flase Positive) is called a false positive, and is judged as correct by apology. The last one is called FN (False Negative), which is apologetically judged as not being correct.

This same idea of a quadrant table is necessary for hypothesis testing. As an example, if there is a data (999 out of 1000) that shows that most of the neurotic patients who went to hot spring cures were cured, the probability would be 99.9%, and we would be tempted to judge that “hot spring cures are effective for neurosis. On the other hand, if there is data that 999 out of 1000 people who did not go to a hot-spring cure also got cured spontaneously, the probability would be as follows

The first hypothesis, “hot water cure is effective for neurosis,” is not a good hypothesis, because the cure occurs with or without hot water cure (or some other factor).

Thus, in order to test a hypothesis, it is necessary to carefully select those that are not (counter-evidence) and conduct a verification experiment.