A “Bernoulli trial” is an experiment or case where the outcome follows a Bernoulli distribution. Some common examples of Bernoulli trials include:

1) A single flip of a coin that may have a head(0) or a tail(1) outcome.

2) A single birth of either a boy(0) or a girl(1).

Bernoulli Trial

For a Bernoulli distribution characterizing a random variable $$ X $$ with probability $$ p $$ for $$ X=1 $$ ("success") and $$ 1-p $$ for $$ X=0 $$ ("failure"), the key characteristics are:
1) Probability mass function: $$ P(X) = p^{X}(1-p)^{1-X} $$
2) Expectation: $$ E[X] = p(1) + (1-p)(0) = p $$
3) Variance: $$ Var(X) = E[X^2] - (E[X])^2 = p - p^2 = p(1-p) $$

Characteristics of Bernoulli Distribution

Bernoulli distribution is a distribution over a single binary variable. It can only have results in 0 or 1. The sum of the probability of 0  and the probability of 1 is equal to 1. If we denote the probability of 0 as $\sigma$, Bernoulli distribution then has a mean of $\sigma$ and a variance of  $\sigma * (1- \sigma)$.

Johns Hopkins University

Claude

Common probability distributions include Normal Distribution, Multinoulli Distribution,  Bernoulli Probability Distribution, Exponential Distribution, and Laplace Distribution. 

Common Probability Distributions

Multi-label classification is a machine learning problem focused on predicting classes that are not mutually exclusive, allowing a single example to be assigned to multiple categories simultaneously. This formulation is particularly useful for auto-tagging tasks, where an item (such as a blog post or an image) might be associated with several correlated tags at once.

Multi-Label Classification

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

Dive into Deep Learning

An empirical distribution is a probability distribution that places an equal probability mass of $$\frac{1}{m}$$ on each of the $$m$$ data points in a given dataset. For continuous random variables, it is defined using the Dirac delta function as $$\hat{p}(x) = \frac{1}{m} \sum_{i=1}^{m} \delta(x-x^{(i)})$$. For discrete random variables, a Multinoulli distribution is sufficient to model the empirical distribution of the observed frequencies.

Empirical Distribution

Mixture distribution is a combination of several other probability distributions.

Mixture Distribution


There are some functions used while describing probability distribution in deep learning models they are
- Logistic Sigmoid
- Softplus Function


Properties of Common Functions

A prior probability distribution is a probability distribution over the parameters of a model that encodes our beliefs about what models are reasonable, before we have seen any data.

prior probability distribution

A normal distribution, also known as a Gaussian distribution, is a continuous probability distribution defined by its mean $$\mu$$ and variance $$\sigma^2$$ (where $$\sigma$$ is the standard deviation). The probability density function is given by the formula: $$p(x) = \frac{1}{\sqrt{2 \pi \sigma^2}} \exp\left(-\frac{1}{2 \sigma^2} (x - \mu)^2ight)$$. Changing the mean $$\mu$$ corresponds to a shift of the distribution along the $$x$$-axis, while increasing the variance $$\sigma^2$$ spreads the distribution out and lowers its peak.

Gaussian (Normal) Distribution

To specify a probability distribution where all mass is concentrated at a single point, the probability density function is defined using the Dirac delta function, $$\delta(x)$$. The Dirac delta function evaluates to zero everywhere except at $$x=0$$, and it integrates to one. A Dirac distribution centered at $$\mu$$ is given by the formula $$p(x) = \delta(x-\mu)$$. This indicates that $$p(x)$$ is infinitely large at $$x=\mu$$ and has a unit area.

Dirac Distribution

The curve of the probability density function for the exponential distribution has a sharp bend at $$x=0$$. The probability density function is given by $$p(x;\lambda)=\lambda \mathbf{1}_{x\geq0} e^{-\lambda x}$$, where $$\mathbf{1}_{x\geq0}$$ is an indicator function that assigns zero probability for all negative values of $$x$$.

Probability Density Function of the Exponential Distribution

The Laplace distribution, also known as the double exponential distribution, is a probability distribution that allows us to place a sharp peak of probability mass at an arbitrary point $$\mu$$. Its probability density function is given by the formula: $$\text{Laplace}(x; \mu, \gamma) = \frac{1}{2 \gamma} \exp\left( - \frac{\lvert x - \mu \rvert}{\gamma} \right)$$.

Laplace Distribution

A simplified example of Multi-Label Classification is photo classification. This is where a given photo may have multiple objects in the scene and a model may predict the presence of multiple known objects in the photo, such as “bicycle,” “apple,” “person,” etc.

Example of Multi-Label Classification

•	Binary Relevance (first-order)
•	Classifier Chains (high-order)
•	Calibrated Label Ranking (second-order)

Problem Transformation Methods of Multi-Label Classification

Algorithm adaptation is a way to directly apply the existing single label algorithm to multiple labels, mainly including:

Multi-Label KNN：

For each instance, firstly, obtain the K instances closest to it, and then use the label set of these instances to judge the value of the predicted label set of this instance through the maximum a posteriori probability (map).

Multi-Label Decision Tree：

The core of using the decision tree to deal with multi-label content is to give more fine-grained information entropy gain criteria to build this decision tree model.

Learn Before

Related

Learn After