Note on probability and statistics

一些定义

Variance \[ Var(X)=D(X)=E((X-E(X))^2)=E(X^2)-E^2(X) \]

Covariance \[ Cov(X,Y)=E((X-E(X))(Y-E(Y)))=E(XY)-E(X)(Y) \]

特别地 \[ Cov(X,X)=Var(X) \] Corelation coefficient \[ \rho_{XY}=\frac{Cov(X,Y)}{\sqrt{Var(X)}\sqrt{Var(Y)}} \] The \(n\)th origin moment \[ E(X^k) \] The \(n\)th central moment \[ E((X-E(X))^k) \] The \((k+l)\)th mixed origin moment \[ E(X^kY^l) \] The \((k+l)\)th mixed central moment \[ E[(X-E(X))^k(Y-E(Y))^l] \]

多元随机变量

\(\textbf{X}=(X_1,X_2,\cdots,X_n)^T\),则有\(E(\textbf{X})=(E(X_1),E(X_2),\cdots,E(X_n))^T\)

定义\(\textbf{X}\)的协方差矩阵 \[ \Sigma=Cov(\textbf{X})\\ =E((\textbf{X}-E(\textbf{X}))(\textbf{X}-E(\textbf{X}))^T)\\ =\begin{bmatrix} Var(X_1)& Cov(X_1,X_2) & \cdots &Cov(X_1,X_n) \\ Cov(X_2,X_1)&Var(X_2) &\cdots &Cov(X_2,X_n) \\ \vdots& \vdots & \ddots & \vdots\\ Cov(X_n,X_1)& Cov(X_n,X_2) &\cdots &Var(X_n) \end{bmatrix} \] \(n\)元正态分布的概率密度为 \[ f(\textbf{x})\\ =f(x_1,x_2,\cdots,x_n)\\ =\frac{1}{(2\pi)^{\frac{n}{2}}|\Sigma|^{\frac{1}{2}}}e^{-\frac{(\textbf{x}-E(\textbf{X}))^T\Sigma^{-1}(\textbf{x}-E(\textbf{X}))}{2}} \] 服从\(n\)元正态分布的随机变量\(\textbf{X}\)中的分量\(X_1,X_2,\cdots,X_n\)相互独立

\(\Leftrightarrow\)它们两两不相关

\(\Leftrightarrow\)\(Cov(\textbf{X})\)为对角矩阵

大数定律

  • Convergence in probability

\[ \lim_{n\rightarrow+\infty}P\{|Y_n-c|<\epsilon\}=1 \]

记为\(Y_n\xrightarrow[]{P}c\)

  • Markov不等式

对任意\(\epsilon>0\),有\(P\{|Y|\ge\epsilon\}\le\frac{E(|Y|^k)}{\epsilon^k}\)

  • Chebyshev不等式

对任意\(\epsilon>0\),有\(P\{|X-\mu|\ge\epsilon\}\le\frac{\sigma^2}{\epsilon^2}\)

  • Khinchin大数定律

\(\{X_i,i\ge1\}\)是独立同分布的随机变量序列,且其数学期望为\(\mu\),则 \[ \lim_{n\rightarrow +\infty}P\{|\frac{1}{n}\sum_{i=1}^{n}X_i-\mu|\ge \epsilon\}=0 \]\[ \frac{1}{n}\sum_{i=1}^{n}X_i\xrightarrow[]{P}\mu \]

一些性质

期望

\[ E(aX)=aE(X)\\ E(X+Y)=E(X)+E(Y) \]

另外,当\(X\)\(Y\)独立等价于\(E(XY)=E(X)E(Y)\)

证明:

  1. \(Y=aX\),所以有\(E(aX)=E(Y)\) \[ F_Y(y)=P\{Y\le y\}=P\{X\le x\}=F_X(x)\\ \therefore f_Y(y)\frac{dy}{dx}=af_Y(y)=f_X(x)\\ \therefore f_Y(y)=\frac{f_X(x)}{a} \] \[ E(aX)=E(Y)=\int_{-\infty}^{+\infty}yf_Y(y)dy=\int_{-\infty}^{+\infty}ax\frac{f_X(x)}{a}dy\\ =\int_{-\infty}^{+\infty}xf_X(x)dy=a\int_{-\infty}^{+\infty}xf_X(x)dx\\ =aE(X) \]

  2. \[ E(X+Y)=\int_{-\infty}^{+\infty}\int_{-\infty}^{+\infty}(x+y)f(x,y)dxdy\\ =\int_{-\infty}^{+\infty}\int_{-\infty}^{+\infty}xf(x,y)dydx+\int_{-\infty}^{+\infty}\int_{-\infty}^{+\infty}yf(x,y)dxdy\\ =\int_{-\infty}^{+\infty}x\int_{-\infty}^{+\infty}f(x,y)dydx+\int_{-\infty}^{+\infty}y\int_{-\infty}^{+\infty}f(x,y)dxdy\\ =\int_{-\infty}^{+\infty}xf_X(x)dx+\int_{-\infty}^{+\infty}yf_Y(y)dy\\ =E(X)+E(Y) \]

\[ E(XY)=\int_{-\infty}^{-\infty}\int_{-\infty}^{-\infty}xyf(x,y)dxdy\\ =\int_{-\infty}^{-\infty}\int_{-\infty}^{-\infty}xyf(x,y)dxdy\\ =\int_{-\infty}^{-\infty}\int_{-\infty}^{-\infty}xyf_X(x)f_Y(y)dxdy\\ =\int_{-\infty}^{-\infty}xf_X(x)dx\int_{-\infty}^{-\infty}yf_Y(y)dy\\ =E(X)E(Y) \]

方差

\[ D(aX)=a^2D(X)\\ D(X+Y)=D(X)+D(Y)+Cov(X,Y) \]

证明:

  1. \[ D(aX)=E[(aX-E(aX))^2]=E[(aX-aE(X))^2]=a^2E((X-E(X))^2)=a^2D(X) \]

  2. \[ D(X+Y)=E[((X+Y)-E(X+Y))^2\\ =E(X^2+Y^2+E^2(X)+E^2(Y)-2XE(X)-2YE(Y)-2XE(Y)-2YE(X)+2XY+2E(X)E(Y))\\ =E(X^2)-E^2(X)+E(Y^2)-E^2(Y)+2(E(XY)-E(X)E(Y))\\ =D(X)+D(Y)+2Cov(X,Y) \]

协方差

\[ Cov(X,Y)=Cov(Y,X)\\ Cov(aX,bY)=abCov(X,Y)\\ Cov(X+Y,Z)=Cov(X,Z)+Cov(Y,Z) \]

证明:

  1. \[ Cov(X,Y)=E[(X-E(X))(Y-E(Y))]=E[(Y-E(Y))(X-E(X))]=Cov(Y,X) \]

  2. \[ Cov(aX,bY)=E[(aX-E(aX))(bY-E(bY))]=abE[(X-E(X))(Y-E(Y))]=abCov(X,Y) \]

\[ Cov(X+Y,Z)=E[(X+Y-E(X+Y))(Z-E(Z))]\\ =E[(X-E(X))(Z-E(Z))+(Y-E(Y))(Z-E(Z))]\\ =E[(X-E(X))(Z-E(Z))]+[(Y-E(Y))(Z-E(Z))]\\ =Cov(X,Z)+Cov(Y,Z) \]

检验假设

如果\(X_i\sim N(\mu_1,\sigma_1^2)(1\le i\le n_1)\)\(Y_i\sim N(\mu_2,\sigma_2^2)(1\le i\le n_2)\),那么有\(\bar{X}\sim N(\mu_1,\frac{\sigma_1^2}{n_1})\)\(\bar{Y}\sim N(\mu_2,\frac{\sigma_2^2}{n_2})\)

  1. \(Z=\frac{\bar{X}-\bar{Y}-\delta}{\sqrt{\frac{\sigma_1^2}{n_1}+\frac{\sigma_2^2}{n_2}}}\sim N(0,1)\),其中\(\delta=\mu_1-\mu_2\)

证明: \[ \because \bar{X}-\bar{Y}-\delta \sim N(0,\frac{\sigma_1^2}{n_1}+\frac{\sigma_2^2}{n_2})\\ \therefore Var(Z)=\frac{1}{\frac{\sigma_1^2}{n_1}+\frac{\sigma_2^2}{n_2}}Var(\bar{X}-\bar{Y}-\delta)=1\\ \therefore Z\sim N(0,1) \]

  1. \(T=\frac{\bar{X}-\bar{Y}-\delta}{S_{\omega}\sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}\sim t(n_1+n_2-2)\),其中\(\delta=\mu_1-\mu_2\)\(S_\omega^2=\frac{(n_1-1)S_1^2+(n_2-1)S_2^2}{n_1+n_2-2}\)并且\(\sigma_1=\sigma_2=\sigma\)

证明: \[ T=\frac{\bar{X}-\bar{Y}-\delta}{S_{\omega}\sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}\\ =\frac{Z}{S_\omega\sqrt{\frac{1}{n_1}+\frac{1}{n_2}}/\sqrt{\frac{\sigma_1^2}{n_1}+\frac{\sigma_2^2}{n_2}}}\\ \because \sigma_1=\sigma_2=\sigma\\ \therefore T=\frac{Z}{S_\omega/\sigma} \] 我们考虑\(\frac{S_\omega}{\sigma}\) \[ \frac{S_\omega}{\sigma}=\sqrt{\frac{S_\omega^2}{\sigma^2}}=\sqrt{\frac{(n_1-1)S_1^2+(n_2-1)S_2^2}{(n_1+n_2-2)\sigma^2}}\\ =\sqrt{(\frac{(n_1-1)S_1^2}{\sigma_1^2}+\frac{(n_2-1)S_2^2}{\sigma_2^2})/(n_1+n_2-2)} \] 不妨设 \[ \chi^2=\frac{(n_1-1)S_1^2}{\sigma_1^2}+\frac{(n_2-1)S_2^2}{\sigma_2^2} \] 显然有\(\chi^2\sim \chi^2(n_1+n_2-2)\)

所以 \[ T=\frac{Z}{\sqrt{\chi^2/(n_1+n_2-2)}}\sim t(n_1+n_2-2) \]

  1. \(T=\frac{\bar{X}-\bar{Y}-\delta}{\sqrt{\frac{S_1^2}{n_1}+\frac{S_2^2}{n_2}}}\left\{\begin{matrix}\sim t(k) & otherwise\\ \sim N(0,1) & n_1,n_2>50 \end{matrix}\right.\),其中\(k=min\{n_1-1,n_2-1\}\)

  2. \(F=\frac{S_1^2}{S_2^2}\sim F(n_1-1,n_2-1)\),其中\(\sigma_1=\sigma_2=\sigma\)

证明: \[ F=\frac{S_1^2}{S_2^2}=\frac{\frac{(n_1-1)S_1^2}{\sigma^2}/(n_1-1)}{\frac{(n_2-1)S_2^2}{\sigma^2}/(n_2-1)}\\ \] \(\because \frac{(n_1-1)S_1^2}{\sigma^2}\sim \chi^2(n_1-1)\)并且\(\frac{(n_2-1)S_2^2}{\sigma^2}\sim \chi^2(n_2-1)\)

所以 \[ F\sim F(n_1-1,n_2-1) \]