\documentclass{article}

\usepackage{LectureNotes}

\begin{document}
\pagestyle{empty}

\begin{centering}
\large
Intro to Econometric Theory \linebreak
Heinz School, Carnegie Mellon University \linebreak
90-906, Spring 2005 \linebreak
\vspace{0.2in} \linebreak
Solution \#2
\normalsize

\end{centering}
\vspace{0.3in}

\begin{enumerate}
\item \label{q:reg-covariance}
      \textbf{
      You may recall that, in the bivariate regression model,
      }

      \begin{equation}
        Y = \beta_1 + \beta_2X_2 + \epsilon
      \end{equation}

      \textbf{
      the OLS estimate of $\beta_2$ is 
      $\hat{\beta}_{2,OLS} = \frac{\widehat{Cov}(Y,X_2)}{\widehat{V(X_2)}}$.
      This means that if the correlation (or covariance) between $Y$
      and $X_2$ is positive, then $\hat{\beta}_2$ is positive.  
      }

      \textbf{
      Consider now a slightly more complex model:
      }

      \begin{equation}
        Y = \beta_1X_1 + \beta_2X_2 + \epsilon
      \end{equation}

      \textbf{
      Let's suppose that $\overline{Y}=\overline{X_1}=\overline{X_2}=0$.
      Please calculate an expression for $\hat{\beta}_{1,OLS}$ and 
      $\hat{\beta}_{2,OLS}$ in terms of the variances and covariances
      of $Y,X_1,X_2$.  Using your formula, discuss whether or not it
      can happen that $\widehat{Cov}(Y,X_1)>0$ while 
      $\hat{\beta}_{1,OLS}<0$.  If it cannot, why not?  If it can,
      then how?  Your goal is both to construct a correct argument
      and to explain in \emph{intuitive} English why your result
      holds.
      }\vspace{0.5\baselineskip}

      The formula for OLS says:

      \begin{eqnarray}
      \left( \begin{array}{c} 
             \hat{\beta}_{1,OLS}\\ \hat{\beta}_{2,OLS} 
             \end{array}
      \right)
      &=&
      \left[ \begin{array}{c c}
             \sum X_1^2  & \sum X_1X_2 \\
             \sum X_2X_1 & \sum X_2^2
             \end{array}
      \right]^{-1}
      \left( \begin{array}{c}
             \sum YX_1 \\ \sum YX_2
             \end{array}
      \right) \nonumber\\
      & & \nonumber\\
      & & \nonumber\\
      &=&
      \left[ \begin{array}{c c}
             \widehat{V(X_1)}       & \widehat{Cov}(X_1,X_2) \\
             \widehat{Cov}(X_1,X_2) & \widehat{V(X_2)}
             \end{array}
      \right]^{-1}
      \left( \begin{array}{c}
             \widehat{Cov}(Y,X_1) \\ \widehat{Cov}(Y,X_2)
             \end{array}
      \right) \nonumber\\
      & & \nonumber\\
      & & \nonumber\\
      &=&
      \frac{1}{\widehat{V(X_1)}\widehat{V(X_2)} - 
               \widehat{Cov}(X_1,X_2)^2} *
      \nonumber\\
      & & \nonumber\\
      & &
      \left[ \begin{array}{c c}
             \widehat{V(X_2)}       & -\widehat{Cov}(X_1,X_2) \\
             -\widehat{Cov}(X_1,X_2) & \widehat{V(X_1)}
             \end{array}
      \right]
      \left( \begin{array}{c}
             \widehat{Cov}(Y,X_1) \\ \widehat{Cov}(Y,X_2)
             \end{array}
      \right) \nonumber\\
      & & \nonumber\\
      & & \nonumber\\
      &=&
      \frac{1}{\widehat{V(X_1)}\widehat{V(X_2)} - 
               \widehat{Cov}(X_1,X_2)^2} *
      \nonumber\\
      & & \nonumber\\
      & &
      \left( \begin{array}{c}
             \widehat{Cov}(Y,X_1) \widehat{V(X_2)} -
             \widehat{Cov}(Y,X_2) \widehat{Cov}(X_1,X_2) \\ 
             \widehat{Cov}(Y,X_2) \widehat{V(X_1)} -
             \widehat{Cov}(Y,X_1) \widehat{Cov}(X_1,X_2)
             \end{array}
             \nonumber
      \right)
      \end{eqnarray}

      Between the first and second equals signs I multiply the $(X'X)^{-1}$
      matrix by N and the $X'Y$ matrix by $1/N$.

      Since $X'X$ is positive definite, its determinant (the fraction above)
      is positive; this result is in the book, pg 47.  So, 
      $\hat{\beta}_{1,OLS}$ will have the same sign as
      $\widehat{Cov}(Y,X_1) \widehat{V(X_2)} -
       \widehat{Cov}(Y,X_2) \widehat{Cov}(X_1,X_2)$.  By looking at the 
      first term, we can see that one factor tending to make  
      $\hat{\beta}_{1,OLS}$ positive would be $\widehat{Cov}(Y,X_1)$
      positive, since $\widehat{V(X_2)}$ must be positive.  

      However,
      it is possible for this factor to be overturned by the second 
      factor in the sum, $-\widehat{Cov}(Y,X_2) \widehat{Cov}(X_1,X_2)$.
      If, for example, the covariance between $Y$ and $X_2$ is also 
      positive and the covariance between $X_1$ and $X_2$ is positive
      then it could happen that $\hat{\beta}_{1,OLS}$ is negative.

      So, $\hat{\beta}_{1,OLS}$ could be negative even though the 
      covariance between $Y$ and $X_1$ is positive if there is a 
      strong positive relationship between $Y$ and $X_2$ and 
      between $X_1$ and $X_2$.  This would happen if all three of
      $Y,X_1,X_2$ moved together, obviously.  This might happen, for
      example, if increases in $X_2$ ``drive'' both increases in
      $Y$ and $X_1$ but $X_1$ itself has a negative influence on
      $Y$.

      Consider the following example.  The covariance (across
      countries or over time) between life expectancy and the
      consumption of red meat per capita is positive. Thinking
      of life expectancy as $Y$ and meat/capita as $X_1$ and
      income as $X_2$, you can see how this kind of situation can
      come about (assuming for purposes of argument we believe that 
      red meat kills you prematurely).  People often call a variable
      playing the role of $X_2$ here a ``confounder'' because it 
      hides the true relationship between $Y$ and $X_1$.
      \vspace{0.5\baselineskip}

\item \textbf{Do problem 9 on page 115 of Greene}\vspace{0.5\baselineskip}

      The easiest method here is the LR-like method.  We will compare the
      models on pages 2 and 3 of the attached output using the 
      F-statistic:

      \begin{eqnarray}
      \mathrm{F-stat} &=& \frac{ (\mathrm{SSE}_{R}-\mathrm{SSE_{UR}})/%
                                        \mathrm{rank(R)} }
                               { \mathrm{SSE}_{UR} / (N-K) }
                          \nonumber\\
                      &=& \frac{ (0.15888-0.03384) / 3}
                               { 0.03384/(36-5)}
                          \nonumber\\
                      &=& 38.2 \nonumber
      \end{eqnarray}

      We compare this to the F table with 3 numerator and 31 denominator
      degrees of freedom.  For a significance level of 1\%, the table
      value is about 4.5.  So, we reject the null.  Prices do affect
      consumption of gas --- hooray for the law of demand.
      

\item \textbf{Do problem 10 on page 115 of Greene --- in class I said that
      you need only test the linear restriction $\gamma_nc=\gamma_uc$}

      Again, the easiest way to do this is with an LR-like test, so
      we compare the regressions on pages 4 and 6.  Think about how I
      have imposed the parameter restriction in this model --- can this
      be done for \emph{any} linear restriction?

      Anyway, comparing these models using an F statistic gives:

      \begin{eqnarray}
      \mathrm{F-stat} &=& \frac{ (\mathrm{SSE}_{R}-\mathrm{SSE_{UR}})/%
                                        \mathrm{rank(R)} }
                               { \mathrm{SSE}_{UR} / (N-K) }
                          \nonumber\\
                      &=& \frac{ (0.0889-0.00856) / 1}
                               { 0.00856/(36-10)}
                          \nonumber\\
                      &=&  1.00 \nonumber
      \end{eqnarray}

      Looking at the F table with 1 numerator and 26 denominator
      degrees of freedom with a 5\% significance level yields a
      table value of about 7.77.  So, we accept the null --- there
      is not enough evidence in the data to reject this null 
      hypothesis.

      We could also pretty easily have done this test just using
      the regression on page 4-5 using a t statistic:

      \begin{eqnarray}
      \mathrm{t-stat} &=& \frac{\widehat{\gamma}_{nc} - 
                                \widehat{\gamma}_{uc}}
                               { \sqrt{
                                  \hat{V}(\hat{\gamma}_{nc} - 
                                   \hat{\gamma}_{uc})} } 
                           \nonumber\\
                      &=& \frac{ 0.08313-(-0.11499) }
                               { \sqrt{
                                  0.04301 + 0.0045 - 2(0.0038)} }
                           \nonumber\\
                      &=& \frac{0.19812}{.200} \nonumber\\
                      &=& 0.99 \nonumber
      \end{eqnarray}

      Obviously, a t-stat this small will not result in a hypothesis
      rejection.  Notice also that this t-stat is the square root
      of the F-stat, as theory demands.

\end{enumerate}

\end{document}