Maximum Likelihood Estimation with Gaussians

Joseph Gonzalez

The Guassian Distribution

The Gaussian Density function

p(x|μ,(σ^2)) = (1/((Sqrt[2π ])σ))Exp[-((((x-μ))^2)/(2(σ^2)))]

A plot:

Manipulate[
Plot[(1/((Sqrt[2π ])σ))Exp[-((((x-μ))^2)/(2(σ^2)))],{x,-5, 5}, PlotRange->All],{{μ,0},-5,5}, {{σ,1},.001,5}, SaveDefinitions->True]

Log is Monotonic Increasing

A Plot of log of x.  Notice that it is monotonicly increasing.

Plot[Log[x], {x, 0, 10}, AxesLabel->{"x", "Log[x]"}]

Observe how the log function perserves the global optimum.

GraphicsRow[{
Plot[-(((x+1))^2)+5, {x, -6, 6}, PlotLabel->"f(x)"],
Plot[Log[-(((x+1))^2)+5], {x, -6, 6},PlotLabel->"Log[f(x)]"]}]

Convex Function

Here I have plotted the log likelihood surface as a function of μ and σ.  You can change the μ and σ of the underlying data.

Manipulate[
Module[{x,n},
n = 30;
x = RandomReal[NormalDistribution[μ,σ],n];
Plot3D[-(n/2)Log[2π] - (n/2)Log[σ] - (∑_(i=1)^n)((((x[[i]] -μ))^2)/(2(σ^2))),{μ,-3,5},{σ,0.01,20},
PlotPoints->30, AxesLabel->{"μ", "σ", "Log Likelihood"}]
], {{μ,0},-5,5}, {{σ,1},.001,5}, SaveDefinitions->True]