\documentclass{article}

%\usepackage{LectureNotes}
\usepackage{amsmath,amssymb}

\begin{document}
\pagestyle{empty}

\begin{centering}
\large
Intro to Econometric Theory \linebreak
Heinz School, Carnegie Mellon University \linebreak
90-906, Spring 2004-5 \linebreak
\vspace{0.2in} \linebreak
Solution \#1
\normalsize

\end{centering}
\vspace{0.3in}

\begin{enumerate}
\item \textbf{
      Consider a sample of people who were asked to report their
      sex (Everyone answers, and everyone answers either male or
      female).  From their answers, we construct the following 
      matrix:
      }

      \begin{equation}
      X = \left[ \begin{array}{c c c} \mathbf{1} & M & F \end{array} \right]
      \end{equation}

      \textbf{
      In the matrix, each row corresponds to one person's response;
      $\mathbf{1}$ is a column of 1s, $M$ is a column with a 1 in
      the $ith$ row if the $ith$ person answered male and a zero 
      otherwise, and $F$ is a column with a 1 in the $ith$ row if
      person $i$ answered female and a zero otherwise.
      }

      \textbf{
      Is $X'X$ invertible.  Why or why not?
      }

      No, $X'X$ is not invertible.  Notice that $M+F=\mathbf{1}$.  This
      is a linear dependency in the columns of $X$.  Thus, the rank of
      $X$ must be less than three.  Since the rank of $X'X$ (a 
      $3 \times 3$ matrix) is the same as the rank of $X$, its rank 
      must be less than 3.  Thus, it is not full rank.  Thus it is not
      invertible.

\item %
      \textbf{
      We discussed in class that variance matrixes are always 
      positive semi-definite but not always positive definite.
      In the case of scalar random variables, a singular 
      variance matrix means a zero variance --- that is, that
      the random variable is not really random at all, but fixed.
      }

      \textbf{
      Now, let's think about what a positive semi-definite but
      not positive definite (hereafter PSDNPD) 
      variance matrix means for the random
      vector it corresponds to.  Let x be a random vector with
      PSDNPD variance matrix V.
      }

      \begin{enumerate}
      \item %
           \textbf{
            Must there be a fixed non-zero vector $\alpha$, such that
            $\alpha'V\alpha$=0?  Why or why not?
            }

       Yes.  If there were no such vector, then $V$ would be
       positive definite, and we are told that it is not.



      \item %
           \textbf{
            Suppose there is such an $\alpha$, what would the variance
            of $\alpha'x$ be?
           }

      Zero.  $V(\alpha'x)=\alpha 'V \alpha=0$.

      \item 
           \textbf{
           What does this tell us about $\alpha'x$?
           }

      It is fixed, non-random.

      \item %
           \textbf{
           Does this mean that $\alpha'x=0$?
           }

      No, it could just as easily be 8 or 0.73 or any fixed number.
      The variance of 0 is zero, but the variance of 8 or 0.73 are
      is zero.

      \item %
           \textbf{
           So, what does it mean for a random vector to have a
            PSDNPD variance matrix?
           }

      When a random vector has a PSDNDP variance matrix, this means
      that there are, in some sense, fewer random variables in the 
      vector than appear to be.  Consider a scalar random variable 
      $z$ and a random vector:
 
      $x=\left( \begin{array}{c} z \\ -z \end{array} \right)$

      As you can see, although $x$ has two elements it does not 
      \emph{really} contain two random variables, just $z$.

      The variance matrix of $x$ is:

      $V(x) = \left[ \begin{array}{c c} 
                     \sigma^2_z & -\sigma^2_z \\
                     -\sigma^2_z & \sigma^2_z
                     \end{array}
              \right]$

      This matrix is singular, and:

      $\left( \begin{array}{c c} 1 & 1 \end{array} \right)
       \left[ \begin{array}{c c} 
              \sigma^2_z & -\sigma^2_z \\
              -\sigma^2_z & \sigma^2_z
             \end{array}
       \right]
      \left( \begin{array}{c} 1 \\ 1 \end{array} \right) = 0$

      and:

      $V\left( (1 \qquad 1)x \right) = 0$

      Another way of saying the same thing is that if V is 
      PSDNPD, then you can always write one of the elements of 
      $x$ as a linear function of the other elements plus a 
      fixed number (this follows from $V(\alpha'x)=0)$.

      \end{enumerate}


\item %
      \textbf{
      Suppose we have an estimator of per-capita income in the US,
      $\hat{I}$, and an estimator of the share of income saved
      (NOT the average over people of share of income saved but
      rather the average over dollars of share of income saved!),
      $\hat{s}$.  The true values of these parameters are
      $\mu_I$ and $\mu_s$. 
      These two estimators are derived somehow from a 
      survey of size $n$, and we have proved the following result for
      these estimators:
      }

      \begin{equation}
      \sqrt{n} \left( \left( \begin{array}{c} 
                             \hat{I}\\ \hat{s}
                             \end{array} 
                      \right) -
                      \left( \begin{array}{c}
                              \mu_I\\ \mu_s 
                             \end{array} 
                      \right)
               \right)
       \overset{A}{\displaystyle{\sim}}
       N \left( \left( \begin{array}{c} 0\\ 0 \end{array} \right) ,
         \left[ \begin{array}{r r}
                400 & 20\\
                 20 &  9
                \end{array}
         \right]
         \right)
      \end{equation}

      \textbf{
      Please construct an approximate (why approximate?) 
      95\% confidence interval for the per-capita savings, $\mu_I \mu_s$.
      }

      Approximate because we are using asymptotic (or approximate) 
      distributions, not the true distributions of the estimators to
      construct the CI --- this is true even though the problem I gave 
      did not contain a statement about asymptotic distribution since 
      the theorem we will use has only a conclusion for asymptotic
      distribution.  The CI is right if the sample is very large,
      and (we hope!) pretty close in sample sizes we are likely to
      encounter.

      Anyway, we have a result from class which says:

      If 
      $\sqrt{n}\left( x-\mu_x \right) 
       \overset{A}{\displaystyle{\sim}} N(0,\Sigma)$ then 
      $\sqrt{n} \left( g(x)-g(\mu_x) \right) 
       \overset{A}{\displaystyle{\sim}}
       N \left( 0, \triangledown _g' \Sigma \triangledown _g \right)$
      In this formula, let's set 
      $x=\left( \begin{array}{c} \hat{I} \\ \hat{s} \end{array} \right)$
      and
      $g(x) = x_1x_2$.  That means that 
      $\triangledown_g = \left( \begin{array}{c} x_2 \\ x_1 
                                \end{array} \right)$
      or
      $\triangledown_g = \left( \begin{array}{c} \hat{s} \\ \hat{I} 
                                \end{array} \right)$

      Now, applying the theorem, we get:

      \begin{eqnarray}
      \sqrt{n} \left( \left( \begin{array}{c} \hat{I}\hat{s} 
                             \end{array} 
                      \right)
                      -
                      \left(\left( \begin{array}{c} \mu_I\mu_s 
                                   \end{array}
                            \right) 
                      \right)
                \right)
       & = &
      \sqrt{n} \left( \left( g(x) 
                      \right)
                      -
                      \left( g(\mu)
                      \right)
                \right)
       \nonumber\\
       & \overset{A}{\displaystyle{\sim}} &
       N \left( 0,
               \left( \begin{array}{c c} \hat{s} & \hat{I} \end{array} \right)
               \left[ \begin{array}{l l} 400 & 20\\
                                          20 &  9
                      \end{array}
               \right]
               \left( \begin{array}{c} \hat{s} \\ \hat{I} \end{array} \right)
          \right)
       \nonumber\\
       &=&
       N \left( 0, 400 \hat{s}^2 + 40 \hat{s} \hat{I} + 9 \hat{I}^2
         \right)
       \nonumber
      \end{eqnarray}

      Let's call the rather messy quadratic $V$.  Had we been given the
      values of $\hat{I}$ and $\hat{s}$, we could have calculated $V$.
      Then, our 95\% confidence interval would be:

      $\hat{I}\hat{s} \pm 1.96 V$


\item \textbf{
      You are interested in the effect of going to the emergency room on
      a person's life expectancy.  There is a sample of people collected
      on a particular day, some of whom went to an emergency room and
      some of whom did not.  They are followed over time until they die.
      From each person there are two items in the dataset: $T_i$ the
      length of time from the day of the survey until death and 
      $ER_i$ a variable which equals one if they went to the emergency room
      and zero if they did not.  You wish to run a regression, but the
      people who conducted the study will not let you have the data, only
      summary statistics.  What should you ask for and how will you use
      it? \label{q:er}
      }

      I can run the regression using only $\overline{T}$, $\overline{ER}$,
      $\widehat{\sigma^2_T}$, $\widehat{\sigma^2_{ER}}$, and 
      $\widehat{Cov}(T,ER)$.  Actually, if I only want to regress $T$ on
      $ER$, I could do without $\widehat{\sigma^2_{ER}}$.  Consider the 
      regression model $T=\beta_1+\beta_2ER+\epsilon$.  I will use the
      sample statistics to calculate the regression coefficients as
      follows:

      \begin{align}
        \hat{\beta}_{2,OLS}  &= \frac{\widehat{Cov}(T,ER)}{\widehat{\sigma^2_{ER}}}
                             \nonumber\\
                             &\\
        \hat{\beta}_{1,OLS}  &= \overline{T}-\hat{\beta}_{2,OLS}\overline{ER}
                             \nonumber
      \end{align}


\item \textbf{
      Considering the example in problem \ref{q:er}, consider two 
      linear models:
      }
      \begin{enumerate}
        \item \label{q:reg-TonER}
              $T_i = \beta_1 + \beta_2ER_i + \epsilon_i$
        \item \label{q:reg-ERonT}
              $ER_i = \beta_3 + \beta_4T_i + \epsilon_i$
      \end{enumerate}
      \textbf{
      Let's call the OLS estimates you would get from regressions run
      on these equations $\widehat{\beta}_{1,OLS},\widehat{\beta}_{2,OLS},
      \ldots$.  What relationship will there be between 
      $\widehat{\beta}_{2,OLS},\widehat{\beta}_{4,OLS}$?  What sign ($\pm$)
      do you expect for $\widehat{\beta}_{2,OLS},\widehat{\beta}_{4,OLS}$?
      \label{q:er-models}  
      }

      Just by examining the formula for OLS estimates, we can see that
      the relationship will be:

      \begin{equation}
        \hat{\beta}_{2,OLS} = \hat{\beta}_{4,OLS}\frac{\widehat{\sigma^2_T}}
                                                      {\widehat{\sigma^2_{ER}}}
      \end{equation}

      Since people who go to the emergency room are probably sick, I expect
      $Cov(T,ER)<0$ and therefore that both $\hat{\beta}_{2,OLS}<0$ and 
      $\hat{\beta}_{4,OLS}<0$.

\item \textbf{
      We usually interpret regression models as embodying some causal
      story (the RHS causing the LHS); however, OLS does not ``know''
      this: it produces estimates regardless of whether our implicit
      causal story is right.  Discuss this fact in the context of the
      models in question \ref{q:er-models} and your expectations about
      the signs of the relevant coefficients.
      }

      Well, the model in question~\ref{q:reg-TonER} would produce the result
      that emergency rooms kill people if we were to take its implicit 
      causal story
      seriously; whereas, the model in question~\ref{q:reg-ERonT} would 
      produce the result that sick people go to the emergency room if we 
      were to take its implicit causal story seriously.  Assuming for the
      moment that we don't believe that emergency rooms kill people, 
      obviously we can go astray if we don't think carefully about what
      causal model we are assuming when we write down and interpret 
      regression models.  That is, if we wrote down, estimated, and interpreted
      the
      model of question~\ref{q:reg-TonER} we would be led to the (false)
      conclusion that emergency rooms kill people; whereas, if we wrote
      down, estimated, and interpreted the model of 
      question~\ref{q:reg-ERonT} we would be led to the (true) conclusion that
      sick people go to emergency rooms.

      Regressions only \emph{use} correlation in making their estimates; 
      therefore, they can not \emph{tell} you about causation.  You must 
      already have a causal theory before you run your regression if you
      want to draw causal conclusions from your results.

\end{enumerate}

\end{document}
