Outline

Respecting the importance of the material in this section, we pause to provide an outline.

• We began by seeking the conditional distribution which had maximal entropy subject to a set of linear constraints (7).
• Following the traditional procedure in constrained optimization, we introduced the Lagrangian , where are a set of Lagrange multipliers for the constraints we imposed on .
• To find the solution to the optimization problem, we appealed to the Kuhn-Tucker theorem, which states that we can (1) first solve for to get a parametric form for in terms of ; (2) then plug back in to , this time solving for .
• The parametric form for turns out to have the exponential form (11).
• The gives rise to the normalizing factor , given in (12).
• The will be solved for numerically using the dual function (14). Furthermore, it so happens that this function, , is the log-likelihood for the exponential model (11). So what started as the maximization of entropy subject to a set of linear constraints turns out to be equivalent to the unconstrained maximization of likelihood of a certain parametric family of distributions.

Table 1 summarizes the primal-dual framework we have established.