Google

Conditional probability is the likelihood of an event occurring, given that another event has already occurred. The event that is given must have a non-zero probability. This relationship is commonly denoted as $P(A|B)$ or $Pr(A|B)$, which reads as 'the probability of A given B'. More generally, it can be represented as $Pr(\cdot|x_1, ..., x_n)$ or simply $p(\cdot|x_1, ..., x_n)$, indicating the probability distribution of a variable conditioned on the values of other variables $x_1, ..., x_n$.

Conditional Probability

Any joint probability can be decomposed into conditional probabilities using Chain Rule. For example: 
$$P(a, b, c) = P(a| b, c) \times P(b, c) =  P(a | b, c) \times P(b | c) \times P(c)$$

Chain Rule

In sequential modeling, autoregressive conditional probability refers to the likelihood of a specific element occurring in a sequence given all the elements that appeared before it. For a sequence of observations at time steps $$t$$, it is the probability distribution over the subsequent value $$x_t$$ given the history of previous values, formally expressed as the conditional probability $$P(x_t \mid x_{t-1}, \ldots, x_1)$$.

Autoregressive Conditional Probability

Conditional probability models are often expressed using the general notation $Pr(\cdot|x_1, ..., x_n)$, which represents the probability distribution of a target variable (indicated by the dot) given a set of conditioning variables $x_1, ..., x_n$. An alternative, more concise notation for this probability distribution is simply $p(\cdot|x_1, ..., x_n)$. This flexible notation is fundamental in statistics and machine learning for defining models that predict outcomes based on various inputs.

General Notation for Conditional Probability Models

The specific prediction, ŷ, made by a model is formally determined by an optimization rule that operates on an objective function. For instance, in probabilistic classification models, the prediction is the class 'y' that maximizes the conditional probability P(y|x), expressed as ŷ = argmax_y P(y|x). The 'arg' operator (argmax for maximization, argmin for minimization) identifies the input value that achieves the optimal value of the function.

Prediction via Optimization

A data science team is comparing two simple models for a predictive text feature. The goal is to predict the next word in a sequence. They want to know which model is more effective at predicting the word 'urgent' after the phrase 'request is'.

- **Model A** bases its prediction on only the single preceding word (e.g., 'is').
- **Model B** bases its prediction on the two preceding words (e.g., 'request is').

Using the corpus data below, determine which model assigns a higher probability to the word 'urgent' in this context. Show the calculations for both models and state your conclusion.

Predictive Text Model Comparison

Imagine a system analyzing a large collection of English sentences. It calculates two values:

1. The probability that any randomly selected word is 'keys'.
2. The probability that a word is 'keys', given that the preceding word was 'piano'.

Which of these two probabilities would you expect to be higher? Justify your reasoning by explaining what each probability represents and how the context provided by the preceding word influences the likelihood.

Interpreting Conditional Probabilities in Text

Suppose we want to find the probability that a patient avoids the flu ($$A$$) given they received the flu shot ($$FS$$). The difference between conditional probability with the $$do()$$ operator and without is:

*   **With the $$do()$$ operator ($$P(A|do(FS))$$)**: The probability that a typical patient would avoid the flu if they were assigned to get the shot (by intervention, removing personal choice).
*   **Without the $$do()$$ operator ($$P(A|FS)$$)**: The probability that a patient avoided the flu given they were observed to take the flu shot. This represents the observed frequency of flu avoidance among patients who volunteered to get the shot.

Conditional Probability With do() Versus Without

A language model is analyzing a text corpus of 10,000 two-word phrases. The analysis reveals the following counts:

- The word 'deep' is the first word in 400 phrases.
- The word 'learning' is the second word in 250 phrases.
- The specific phrase 'deep learning' occurs 80 times.

Based on this data, what is the probability that the second word of a phrase is 'learning', given that the first word is 'deep'?

Learn Before

Related