Probability.
Probability is a quantification of the degree to which an event is likely to occur; specifically, the probability of an event occurring is calculated by dividing the number of cases in which the event occurs (the preferred outcome) by the overall number of cases (all possible outcomes).
The probability is expressed as a range from 0 to 1. The closer to 0, the less likely the event is to occur, and the closer to 1, the more likely it is to occur. For example, the probability of a coin tossing a coin yielding a face is 0.5 (50%), while the probability of rolling a dice yielding a 1 is 1/6 (approximately 16.7%).
Probability can be estimated based on subjective factors or experience, as in the Bayesian estimation described in ‘Overview and various implementations of Bayesian estimation’, but probability theory takes the approach of using statistical methods and mathematical models to calculate probabilities and elucidate their properties and behaviour.
Probability has several basic properties. For example, the probability of any event is greater than or equal to zero, the probability of the whole event (the certain event) is one, and the probability of the occurrence of two mutually exclusive events (events that cannot occur simultaneously) is the sum of the probabilities of each event (addition theorem). Furthermore, the probability of both of two independent events occurring is the probability of each event multiplied by the probability of each event (multiplication theorem).
Probability is used in a variety of applications: in statistics, probability is used to estimate the probability distribution of data and in testing hypotheses. Probability-based models and algorithms are widely used in machine learning, and the concept of probability also plays an important role in risk assessment, financial engineering, signal processing and game theory.
Probability and Uncertainty
This uncertainty, which is closely related to probability, is often recognised as similar, but has a concept that is strictly different from probability.
Probability, as mentioned above, is a quantification of the degree to which an event is likely to occur, and takes a value between 0 and 1, with the closer to 0, the lower the probability that the event will occur, and the closer to 1, the higher the probability, and is calculated using statistical methods and mathematical models, and the calculation of probability is based on the past occurrence of an event It can be estimated from patterns or data, or based on the nature of the event or assumptions made about it.
In contrast, uncertainty refers to the incompleteness of information and the difficulty of predicting it, and describes a situation in which the exact outcome or outcome of an event cannot be predicted. Uncertainty is caused by factors such as the availability of new information, the presence of unknowns and complex interactions, which cannot be predicted definitively in advance and therefore pose challenges in decision-making and decision-making.
As probability is a mathematically tractable approach, it can be used to quantify and predict uncertain situations and information, and approaches are sometimes taken to quantify and predict uncertainties. In such approaches, probability is used as an indicator or part of a model in dealing with uncertainty, but since probability itself is not uncertainty, the following caution should be exercised when applying probability to uncertainty
- Interpreting probability: probability is a numerical expression of the likelihood of an event occurring, usually in the range 0 to 1. However, how this number is interpreted is important: a high probability does not necessarily mean that the event will occur, and a low probability does not necessarily mean that it will not occur.
- Basis for probability: it is necessary to check how reliable the assumptions and data used to calculate the probability are. If the sample size is small or the data is biased, the probability obtained will also be unreliable.
- Probability and frequency: probability is often interpreted as the frequency over a long period of time, but this is not always observed to be the case for one-off events. For example, in the tossing of a coin, theoretically there is a 50% chance of each side coming out heads or tails, but after a few tosses a bias may occur.
- Overconfidence in probability: it is dangerous to rely too heavily on outcomes just because the probability is high, and factors other than probability need to be carefully considered, especially in high-risk decisions or those with major consequences.
- Probability and risk: probability indicates one aspect of ‘what could happen’, while risk indicates ‘what the consequences of that outcome will be’. Even if the probability is low, the risk should be considered high if the consequences have a significant impact.
- Communicating probabilities: when communicating probabilities to others, care needs to be taken not to mislead, to clearly explain what the probability figures mean and how they should be interpreted, and to avoid misrepresenting uncertainty.
Randomness and Probability
The use of randomness is important when performing the simulations described in ‘Simulation, data science and artificial intelligence’. This randomness and probability are also closely related.
Randomness refers to the property that events are unpredictable and the outcome depends on chance, whereas probability is a quantification of the degree to which an event is likely to occur, and is a means of expressing uncertainty. Randomness may therefore be caused by stochastic factors.
When randomness exists, the outcome of an event is probabilistic and cannot be accurately predicted in advance. For example, when rolling dice, it is not possible to predict which eyes will appear, but the probability that each eye will appear is equal and is expected to be a random outcome. In such cases, it is common to use probabilistic models and probability theory to predict random outcomes.
On the other hand, given a probability, the outcome is determined on the basis of randomness. For example, if a fair coin is tossed, the probability of getting a table is 0.5. In this case, since the probability is given, the outcome is determined on the basis of randomness, but each outcome is independent of each other, which is a predictable randomness.
Thus, probability is a tool for dealing with randomness mathematically, and randomness can also be said to be a property of phenomena and outcomes caused by stochastic elements. The mathematical aspects of probability provide a framework for understanding, predicting and modelling randomness.
Mathematical approach to probability
Probability theory is a mathematical study of uncertain events and phenomena; probability theory uses the concept of probability to describe uncertainty and randomness and to elucidate their nature and behaviour.
Probability is generally calculated by dividing the ‘number of cases in which an event occurs (the favourable outcome)’ by the ‘total number of cases (all possible outcomes)’. Such an approach can be expressed within the framework of set theory, which is a systematic framework for dealing with sets of elements and their combinations.
Specifically, probability theory defines the set of events as a specific set, called the ‘sample space’, and treats subsets of it as ‘events’. For example, if a coin is tossed, the sample space is a set with elements {‘table’, ‘flip’}, and subsets within the sample space, e.g. {‘table’} or {‘flip’}, can be treated as individual events.
This concept of set theory enables the calculation and properties of probabilities to be clearly expressed. For example, probability sums, products and conditional probabilities are understood using the concepts of set sums, products and conditional sets.
Probability theory also uses the concept of probability space, which is a combination of a sample space and its associated probability function, formalised using the ideas of set theory. Probability spaces provide a mathematical framework for defining the probability of events.
Set theory plays an important role as the basis of probability theory and is a tool for clearly defining and manipulating the concept of probability. Expressing the laws and axioms of probability theory in the language of set theory allows for a systematic understanding and application of probability theory, and set theory also plays an important role in the classical probability theory textbooks mentioned in the ‘Introduction to Probability Theory Reading Notes’.
With these mathematical tools, probability theory has laws and axioms for the computation and properties of probability, such as the additive theorem, the multiplicative theorem, conditional probability and Bayes’ theorem, which make it possible to analyse a wide range of probabilistic phenomena and problems.
Probability theory is used in a wide range of applications, including statistics, machine learning, financial engineering, signal processing, communications engineering and game theory. For example, probability theory is closely related to statistics as a method for statistically handling uncertain data and making decisions and predictions, and probability theory also plays an important role in the development and evaluation of machine learning algorithms.
Reference Information and Reference Books
For more information on probabilistic approaches, see “Mathematics in Machine Learning” “Probabilistic Generative Models” and “Machine Learning with Bayesian Inference and Graphical Models,” among others.
For reference books on the theory and history of probability and statistics, see “Probability Theory for Beginners: A Reading Memo” ,”Introduction to Probability Theory: A Reading Memo” ,”Nine Stories of Probability and Statistics that Changed Humans and Society: A Reading Memo” and “134 Stories of Probability and Statistics that Changed the World: A Reading Memo. For specific implementations and applications, see “Statistical Modeling with Python” ,”Statistical Analysis and Correlation Evaluation Using Clojure/Incanter” ,”Probability Distributions Used in Probabilistic Generative Models” etc.
A good reference book on Bayesian estimation is “The Theory That Would Not Die: How Bayes’ Rule Cracked the Enigma Code, Hunted Down Russian Submarines, & Emerged Triumphant from Two Centuries of C“
“Think Bayes: Bayesian Statistics in Python“
“Bayesian Modeling and Computation in Python“
コメント