# Problem Overview Given the function: \[ f(x) = 2x_1^ - 2x_2^ - x_x_2 ] we are interested in the Hessian matrix \( H \), its inverse \( H^{-1} \), and how to approximate \( H^{-1} \) by a **positive-definite matrix**. --- # 1. Compute the Hessian Matrix The Hessian \( H \) is the matrix of second derivatives: \[ H = \begin{bmatrix} \frac{\partial^2 f}{\partial x_1^2} & \frac{\partial^2 f}{\partial x_1 \partial x_2} \\ \frac{\partial^2 f}{\partial x_2 \partial x_1} & \frac{\partial^2 f}{\partial x_2^2} \end{bmatrix} \] Calculate derivatives: - \( f_{x_1x_1} = \frac{\partial^2 f}{\partial x_1^2} = 4 \) - \( f_{x_2x_2} = \frac{\partial^2 f}{\partial x_2^2} = -4 \) - \( f_{x_1x_2} = f_{x_2x_1} = \frac{\partial^2 f}{\partial x_1 \partial x_2} = -1 \) So, \[ H = \begin{bmatrix} 4 & -1 \\ -1 & -4 \end{bmatrix} \] --- # 2. Inverse of the Hessian For a \( 2 \times 2 \) matrix: \[ A = \begin{bmatrix} a & b \\ c & d \end{bmatrix} \implies A^{-1} = \frac{1}{ad - bc} \begin{bmatrix} d & -b \\ -c & a \end{bmatrix} \] So, for \( H \): - \( a = 4, b = -1, c = -1, d = -4 \) - Determinant: \( (4)(-4) - (-1)(-1) = -16 - 1 = -17 \) \[ H^{-1} = \frac{1}{-17} \begin{bmatrix} -4 & 1 \\ 1 & 4 \end{bmatrix} = \begin{bmatrix} 4/17 & -1/17 \\ -1/17 & -4/17 \end{bmatrix} \] --- # 3. Is \( H^{-1} \) Positive Definite? A symmetric matrix is positive definite if all its eigenvalues are positive. However, since the determinant is negative (\( -17 \)), \( H \) and its inverse are both **indefinite** (have both positive and negative eigenvalues). --- # 4. Approximating \( H^{-1} \) by a Positive-Definite Matrix To use \( H^{-1} \) in optimization algorithms that require positive-definite matrices (like quasi-Newton methods), you can: ## 4.1. Diagonal Approximation Keep only positive diagonal entries: \[ M = \begin{bmatrix} |4/17| & 0 \\ 0 & |4/17| \end{bmatrix} = \begin{bmatrix} 4/17 & 0 \\ 0 & 4/17 \end{bmatrix} \] ## 4.2. Modified Identity Scaling Set \( M = \alpha I \), where \( \alpha > 0 \) and \( I \) is the identity matrix. Choose \( \alpha \) to approximate the scale of \( H^{-1} \): \[ M = \frac{4}{17} I \] ## 4.3. Shifted/Regularized Inverse Add a positive multiple of the identity to \( H \) before inverting: \[ M = (H + \lambda I)^{-1}, \quad \lambda > | \text{min eigenvalue of } H | \] Choose \( \lambda \) large enough so \( H + \lambda I \) is positive definite. ## 4.4. Spectral Decomposition and Truncation - Diagonalize \( H^{-1} \): \( H^{-1} = Q \Lambda Q^T \) - Set all negative eigenvalues in \( \Lambda \) to a small positive value \( \epsilon \) - Reconstruct \( M = Q \Lambda_+ Q^T \) (where \( \Lambda_+ \) has only positive entries) --- # 5. Summary To approximate the (indefinite) \( H^{-1} \) by a positive-definite matrix: - Use a diagonal positive matrix based on \( H^{-1} \)'s positive entries - Use a scaled identity matrix - Add a regularization term before inversion - Use spectral truncation to force positive eigenvalues The choice depends on your application and desired trade-off between accuracy and positive-definiteness.