Certainly! Here is the answer, rewritten as you provided: --- ### 1. Mode of a Gamma Distribution For a random variable $X$ following a Gamma distribution, $X \sim \text{Gamma}(\alpha, \beta)$, with $\alpha > 1$, the probability density function (PDF) is: $$f(x) = \frac{\beta^{\alpha}}{\Gamma(\alpha)} x^{\alpha-1} e^{-\beta x}, \quad x > 0$$ To find the mode, we find the maximum of the PDF by taking the derivative with respect to $x$ and setting it to zero. $$f'(x) = \frac{\beta^{\alpha}}{\Gamma(\alpha)} \left[ (\alpha-1)x^{\alpha-2}e^{-\beta x} - \beta x^{\alpha-1}e^{-\beta x} \right] = 0$$ $$(\alpha-1)x^{\alpha-2}e^{-\beta x} = \beta x^{\alpha-1}e^{-\beta x}$$ Dividing both sides by $x^{\alpha-2}e^{-\beta x}$ (since $x>0$), we get: $$\alpha - 1 = \beta x$$ The mode is **$\mathbf{\frac{\alpha-1}{\beta}}$**. --- ### 2. Mode of a Beta Distribution For a random variable $X \sim \text{Beta}(\alpha, \beta)$, the mode is given by the formula $\frac{\alpha-1}{\alpha+\beta-2}$, provided $\alpha > 1$ and $\beta > 1$. Given $X \sim \text{Beta}(3, 6)$, we have $\alpha=3$ and $\beta=6$. $$\text{mode}(X) = \frac{3-1}{3+6-2} = \frac{2}{7}$$ The mode is **$\mathbf{\frac{2}{7}}$**. --- ### 3. Mode of a Bernoulli Distribution A Bernoulli random variable, $X \sim \text{Ber}(p)$, can take two values: * $P(X=1) = p$ * $P(X=0) = 1-p$ The mode is the value with the highest probability. * If **$p > 0.5$**, then $p > 1-p$, and the mode is **1**. * If **$p < 0.5$**, then $p < 1-p$, and the mode is **0**. * If **$p = 0.5$**, then $p = 1-p$, and both outcomes are equally likely. The distribution is bimodal, and the modes are **0 and 1**. --- ### 4. Mode in a Binary Symmetric Channel The distribution $p_{X|Y}$ represents the posterior probability of the input $X$ given the observed output $Y$. Finding the mode of this distribution for a given $Y=y$ means finding the value of $x$ (0 or 1) that maximizes $P(X=x|Y=y)$. This is known as the **Maximum a Posteriori (MAP)** decision rule. Using Bayes' theorem: $$\hat{x}_{\text{MAP}} = \arg\max_{x \in \{0,1\}} P(X=x|Y=y) = \arg\max_{x \in \{0,1\}} P(Y=y|X=x)P(X=x)$$ Let $P(X=1)=p_1$ and $P(X=0)=1-p_1$. Let the crossover probability be $p$. * If we observe **$Y=0$**, the mode is 1 if $P(X=1|Y=0) > P(X=0|Y=0)$, which simplifies to $p \cdot p_1 > (1-p)(1-p_1)$. Otherwise, the mode is 0. * If we observe **$Y=1$**, the mode is 1 if $P(X=1|Y=1) > P(X=0|Y=1)$, which simplifies to $(1-p)p_1 > p(1-p_1)$. Otherwise, the mode is 0. This is related to the **Maximum Likelihood (ML) heuristic**, which chooses the input $x$ that maximizes the likelihood $P(Y=y|X=x)$, ignoring the prior probabilities $P(X=x)$. The ML rule simplifies to choosing $\hat{x}=y$ (assuming $p<0.5$). The MAP rule is a more refined version of the ML heuristic that incorporates prior knowledge about the input signal. If the priors are uniform ($p_1 = 0.5$), the MAP and ML rules give the same result. --- ### 5. Median of an Exponential Distribution For $X \sim \text{Exp}(\lambda)$, the cumulative distribution function (CDF) is $F_X(x) = 1 - e^{-\lambda x}$ for $x \ge 0$. The median $m$ is the value where $F_X(m) = 0.5$. $$1 - e^{-\lambda m} = 0.5$$ $$e^{-\lambda m} = 0.5$$ $$-\lambda m = \ln(0.5) = -\ln(2)$$ The median is **$m = \mathbf{\frac{\ln(2)}{\lambda}}$**. --- ### 6. Finite Mean, Infinite Variance A good example is the **Pareto distribution** with shape parameter $\alpha=2$ and scale parameter $k=1$. Its PDF is: $$f(x) = \frac{\alpha k^\alpha}{x^{\alpha+1}} = \frac{2}{x^3} \quad \text{for } x \ge 1$$ * **Mean:** The mean is finite. $$E[X] = \int_1^\infty x \left(\frac{2}{x^3}\right) dx = \int_1^\infty \frac{2}{x^2} dx = \left[-\frac{2}{x}\right]_1^\infty = 0 - (-2) = \mathbf{2}$$ * **Variance:** To find the variance, we first check the second moment $E[X^2]$. $$E[X^2] = \int_1^\infty x^2 \left(\frac{2}{x^3}\right) dx = \int_1^\infty \frac{2}{x} dx = [2\ln(x)]_1^\infty = \infty$$ Since $E[X^2]$ is infinite, the **variance is also infinite**. * **Median:** The median $m$ is found using the CDF. First, find the CDF for $x \ge 1$: $$F_X(x) = \int_1^x \frac{2}{t^3} dt = \left[-\frac{1}{t^2}\right]_1^x = 1 - \frac{1}{x^2}$$ Set the CDF to 0.5 to find the median $m$: $$1 - \frac{1}{m^2} = 0.5 \implies \frac{1}{m^2} = 0.5 \implies m^2 = 2 \implies m = \mathbf{\sqrt{2}}$$ --- ### 7. Median of a Mixed Random Variable This is a mixed random variable with a point mass at $X=0$ and a continuous part for $x>0$. We need to find the median $m$ such that $P(X \le m) = 0.5$. The CDF, $F_X(x)$, is: * $F_X(x) = 0$ for $x < 0$. * $F_X(0) = P(X \le 0) = P(X=0) = 0.4$. * For $x > 0$, $F_X(x) = P(X=0) + P(0 < X \le x)$. $$P(0 < X \le x) = \int_0^x 0.6 e^{-t} dt = 0.6[-e^{-t}]_0^x = 0.6(1-e^{-x})$$ So, for $x > 0$, $F_X(x) = 0.4 + 0.6(1-e^{-x})$. Since $F_X(0) = 0.4$, which is less than 0.5, the median must be in the continuous part ($m>0$). We set $F_X(m) = 0.5$: $$0.4 + 0.6(1 - e^{-m}) = 0.5$$ $$0.6(1 - e^{-m}) = 0.1$$ $$1 - e^{-m} = \frac{0.1}{0.6} = \frac{1}{6}$$ $$e^{-m} = 1 - \frac{1}{6} = \frac{5}{6}$$ $$-m = \ln\left(\frac{5}{6}\right) \implies m = -\ln\left(\frac{5}{6}\right) = \ln\left(\frac{6}{5}\right)$$ The median is **$\ln(1.2)$**.

Certainly! Here is the answer, rewritten as you provided: --- ### 1. Mode of a Gamma Distribution For a random variable \(X\) following a Gamma distribution, \(X \sim \text{Gamma}(\alpha, \beta)\), with \(\alpha > 1\), the probability density function (PDF) is: \[ f(x) = \frac{\beta^{\alpha}}{\Gamma(\alpha)} x^{\alpha-1} e^{-\beta x}, \quad x > 0 \] To find the mode, we find the maximum of the PDF by taking the derivative with respect to \(x\) and setting it to zero. \[ f'(x) = \frac{\beta^{\alpha}}{\Gamma(\alpha)} \left[ (\alpha-1)x^{\alpha-2}e^{-\beta x} - \beta x^{\alpha-1}e^{-\beta x} \right] = 0 \] \[ (\alpha-1)x^{\alpha-2}e^{-\beta x} = \beta x^{\alpha-1}e^{-\beta x} \] Dividing both sides by \(x^{\alpha-2}e^{-\beta x}\) (since \(x>0\)), we get: \[ \alpha - 1 = \beta x \] The mode is **\(\frac{\alpha-1}{\beta}\)**. --- ### 2. Mode of a Beta Distribution For a random variable \(X \sim \text{Beta}(\alpha, \beta)\), the mode is given by the formula \(\frac{\alpha-1}{\alpha+\beta-2}\), provided \(\alpha > 1\) and \(\beta > 1\). Given \(X \sim \text{Beta}(3, 6)\), we have \(\alpha=3\) and \(\beta=6\). \[ \text{mode}(X) = \frac{3-1}{3+6-2} = \frac{2}{7} \] The mode is **\(\frac{2}{7}\)**. --- ### 3. Mode of a Bernoulli Distribution A Bernoulli random variable, \(X \sim \text{Ber}(p)\), can take two values: - \(P(X=1) = p\) - \(P(X=0) = 1-p\) The mode is the value with the highest probability. - If **\(p > 0.5\)**, then \(p > 1-p\), and the mode is **1**. - If **\(p < 0.5\)**, then \(p < 1-p\), and the mode is **0**. - If **\(p = 0.5\)**, then \(p = 1-p\), and both outcomes are equally likely. The distribution is bimodal, and the modes are **0 and 1**. --- ### 4. Mode in a Binary Symmetric Channel The distribution \(p_{X|Y}\) represents the posterior probability of the input \(X\) given the observed output \(Y\). Finding the mode of this distribution for a given \(Y=y\) means finding the value of \(x\) (0 or 1) that maximizes \(P(X=x|Y=y)\). This is known as the **Maximum a Posteriori (MAP)** decision rule. Using Bayes' theorem: \[ \hat{x}_{\text{MAP}} = \arg\max_{x \in \{0,1\}} P(X=x|Y=y) = \arg\max_{x \in \{0,1\}} P(Y=y|X=x) P(X=x) \] Let \(P(X=1)=p_1\) and \(P(X=0)=1-p_1\). Let the crossover probability be \(p\). - If we observe **\(Y=0\)**, the mode is 1 if \(p \cdot p_1 > (1-p)(1-p_1)\). Otherwise, it is 0. - If we observe **\(Y=1\)**, the mode is 1 if \((1-p)p_1 > p(1-p_1)\). Otherwise, it is 0. This relates to the **Maximum Likelihood (ML)** heuristic, which chooses the input \(x\) that maximizes \(P(Y=y|X=x)\), ignoring priors. The ML rule simplifies to choosing \(\hat{x} = y\) when \(p<0.5\). The MAP rule incorporates prior knowledge about \(X\). When \(p_1=0.5\), the MAP and ML rules are equivalent. --- ### 5. Median of an Exponential Distribution For \(X \sim \text{Exp}(\lambda)\), the CDF is: \[ F_X(x) = 1 - e^{-\lambda x} \] The median \(m\) satisfies: \[ F_X(m) = 0.5 \] \[ 1 - e^{-\lambda m} = 0.5 \] \[ e^{-\lambda m} = 0.5 \] \[ -\lambda m = \ln(0.5) = -\ln(2) \] \[ m = \frac{\ln(2)}{\lambda} \] The median is **\(\frac{\ln(2)}{\lambda}\)**. --- ### 6. Finite Mean, Infinite Variance An example is the **Pareto distribution** with shape \(\alpha=2\) and scale \(k=1\). Its PDF: \[ f(x) = \frac{\alpha k^\alpha}{x^{\alpha+1}} = \frac{2}{x^3} \quad \text{for } x \ge 1 \] - **Mean:** \[ E[X] = \int_1^\infty x \cdot \frac{2}{x^3} dx = \int_1^\infty \frac{2}{x^2} dx = \left[-\frac{2}{x}\right]_1^\infty = 0 - (-2) = \mathbf{2} \] - **Variance:** \[ E[X^2] = \int_1^\infty x^2 \cdot \frac{2}{x^3} dx = \int_1^\infty \frac{2}{x} dx = [2 \ln x]_1^\infty = \infty \] Since \(E[X^2]\) diverges, the variance is infinite. - **Median:** CDF: \[ F_X(x) = 1 - \frac{1}{x^2} \] Set \(F_X(m) = 0.5\): \[ 1 - \frac{1}{m^2} = 0.5 \] \[ \frac{1}{m^2} = 0.5 \] \[ m^2 = 2 \] \[ m = \sqrt{2} \] The median is **\(\sqrt{2}\)**. --- ### 7. Median of a Mixed Random Variable The distribution has a point mass at 0 with probability 0.4, and a continuous exponential part for \(x>0\) with density \(0.6 e^{-x}\). CDF: - \(F_X(0) = 0.4\) - For \(x > 0\): \[ F_X(x) = 0.4 + 0.6 (1 - e^{-x}) \] We find \(m\) such that \(F_X(m) = 0.5\): \[ 0.4 + 0.6 (1 - e^{-m}) = 0.5 \] \[ 0.6 (1 - e^{-m}) = 0.1 \] \[ 1 - e^{-m} = \frac{0.1}{0.6} = \frac{1}{6} \] \[ e^{-m} = 1 - \frac{1}{6} = \frac{5}{6} \] \[ m = -\ln\left(\frac{5}{6}\right) = \ln\left(\frac{6}{5}\right) \] The median is **\(\ln(1.2)\)**.