Hypothesis Test

We have a null hypothesis $H_0$, and an alternative one $H_1$, we'll decide whether we should reject $H_0$ or not
E.g. we are told a coin is fair and we would like to verify if it is so, so we have $H_0: p(h)=0.5$, and $H_1: p(h)\neq 0.5$. $p(h)$ is the probability getting head.
We tossed the coin 10 times, and assume we have 4 heads. Assume coin is fair, what's the probability such events or even skewer events happen by chance?
Let's do the math: p(at most 4 heads out of 10 tosses) = $\frac{1}{2^{10}}(C(10,1)+C(10,2)+C(10,3)+C(10,4)) = 0.37$ Apparently too large to reject.
What about p(at most 3 heads out of 10 tosses) = $\frac{1}{2^{10}}(C(10,1)+C(10,2)+C(10,3)) = 0.17$ Still too large.
What about p(at most 2 heads out of 10 tosses) = $\frac{1}{2^{10}}(C(10,1)+C(10,2)) = 0.05$ Then we are 95% sure this doesn't happen by chance and we can reject null at confidence level of 95%.

Paired?
Parametric or Non-parametric?
Degree of freedom.
Directional? Is it enough to be different? Or we can testing if it's significantly better(or worse)?

Establish the statistic: $\chi^2 = \frac{(O - E)^2}{E}$. $O$ is observed value. $E$ is expected value.
Null hypothesis: coin is fair. Alternative hypothesis: coin is not fair
Observation: tossed coin 10 times, and get 4 heads.
Establish the test-statistic: $\frac{(4-5)^2+(6-5)^2}{5} = 0.4$. Degree of freedom is 1 since there are only two outcomes of a toss. By looking at the significance table (http://sites.stat.psu.edu/~mga/401/tables/Chi-square-table.pdf) we can't reject null.
If we get 3 heads: $\frac{(3-5)^2+(7-5)^2}{5} = 1.6$ we can't reject.
get 2 heads: $\frac{(2-5)^2+(8-5)^2}{5} = 3.6$ we reject at 95% confidence level.
get 1 heads: $\frac{(1-5)^2+(9-5)^2}{5} = 6.4$ we reject at 99% confident level.
How many we need in order to reject for a sample of 20? 4 heads or less to reach 95% confidence level

1-sample, 2-sample, and paired t-test.
1-sample: compare one sample mean to null hypothesis.
2-sample: compare two sample means
paired t-test: compare paired observation (before & after).
Establish the test-statistic: $\frac{(4-5)^2+(6-5)^2}{5} = 0.4$. Degree of freedom is 1 since there are only two outcomes of a toss. By looking at the significance table (http://sites.stat.psu.edu/~mga/401/tables/Chi-square-table.pdf) we can't reject null.
If we get 3 heads: $\frac{(3-5)^2+(7-5)^2}{5} = 1.6$ we can't reject.
get 2 heads: $\frac{(2-5)^2+(8-5)^2}{5} = 3.6$ we reject at 95% confidence level.
get 1 heads: $\frac{(1-5)^2+(9-5)^2}{5} = 6.4$ we reject at 99% confident level.
How many we need in order to reject for a sample of 20? 4 heads or less to reach 95% confidence level

$D = -2\ln\frac{L(null model)}{L(alternative model)} = -2LL(null) - 2LL(alternative)$ and D is Chi-square distributed with df = #params of alternative model - #params of null model
Reading:
- http://vassarstats.net/textbook/ all kinds of tests