Understanding A/B Testing and Statistics Behind



1. A trivial example: flipping a coin

Question:

A coin is tossed for 100 times, heads appeared 65 times, tails appeared 35 times. Construct a hypothesis test to check whether the coin is fair. ($P_{head}=P_{tail}=\frac{1}{2}$)

Hypothesis: $$H_0:P_{h}=0.5, H_a:P_{h}\neq 0.5$$

(1) Z test

Since for binomial distribution, each trial is either 0 or 1, so the sample mean is also the proportion of success. Since for this experiment, we already know the distribution of tossing a coin is a binomial distribution, let $X$ to be the count of successes, then: $$\mu_X = np$$ $$\sigma^2_X=np(1-p)$$ $$X \sim N(np, np(1-p))$$

more ...