\documentclass[12pt]{article}
\usepackage{amsmath}
\usepackage{geometry}
\usepackage{url}
\geometry{top=1in,bottom=1in,left=1in,right=1in}
\setlength{\parindent}{0in}
\setlength{\parskip}{2ex}
\begin{document}
\begin{center}
\begin{tabular*}{6.5in}{l@{\extracolsep{\fill}}r}
\multicolumn{2}{c} {\bfseries Problem Set 12} \\
& \\
ECON 772001 - Math for Economists & Peter Ireland \\
Boston College, Department of Economics & Fall 2019 \\
& \\
\multicolumn{2}{c} {Due Thursday, November 21}
\end{tabular*}
\end{center}
{\bfseries 1. Linear-Quadratic Dynamic Programming}
This problem will give you more practice with dynamic programming under certainty; it presents another case in which an explicit solution for the value function can be found using the guess-and-verify method. The problem is to choose sequences $\{z_{t}\}_{t=0}^{\infty}$ for a flow variable and $\{y_{t}\}_{t=1}^{\infty}$ for a stock variable to maximize the objective function
$$
\sum_{t=0}^{\infty} \beta^{t} (Ry_{t}^2+Qz_{t}^{2}),
$$
subject to the constraints $y_{0}$ given and
$$
Ay_{t}+Bz_{t} \geq y_{t+1}
$$
for all $t=0,1,2,...$, where $\beta$, $R$, $Q$, $A$, and $B$ are constant, known parameters. This problem can be described as being ``linear-quadratic,'' because the constraint is linear and the objective function is quadratic. The discount factor lies between zero and one, $0<\beta<1$, and to make the objective function concave, it is helpful to assume that $R<0$ and $Q<0$ as well.
\begin{description}
\item a. Write down the Bellman equation for this problem.
\item b. Now guess that the value function also takes the quadratic, time-invariant form
$$
v(y_{t};t) = v(y_{t}) = Py_{t}^{2},
$$
where $P$ is an unknown constant. Using this guess, derive the first-order condition for $z_{t}$ and the envelope condition for $y_{t}$.
\item c. Use your results from above to show that the unknown $P$ must satisfy
$$
P = R + \frac{\beta A^{2}QP}{Q+\beta B^{2} P}.
$$
\item d. The equation for $P$ that you just derived for part (c) above is often referred to as an ``algebraic Riccati equation,'' named after the Italian mathematician Jacopo Francesco Riccati who studied equations of this form in the 18th century. There are two approaches that are commonly used to solve Riccati equations. The first recognizes that the equation can be rewritten as
$$
\beta B^{2} P^{2} + [(1-\beta A^{2})Q-\beta B^{2}R]P - QR = 0,
$$
and solved using the quadratic formula. Along those lines, suppose that the specific numerical values $\beta=0.95$, $R=-0.01$, $Q=-1$, $A=1.05$, and $B=-1$ are assigned to the model's parameters, and use the quadratic formula to find the specific numerical value for $P$. In general, the quadratic formula gives two possible solutions for $P$, but for problems like this one it turns out that only one will be negative, implying that the value function, like the objective function from the original problem, is concave; then it can be shown that only the negative value of $P$ corresponds to the true solution to the original dynamic optimization problem.
\end{description}
A second approach to solving the Riccati equation is to convert it into a difference equation of the form
$$
P_{t+1} = R + \frac{\beta A^{2}QP_{t}}{Q+\beta B^{2} P_{t}}.
$$
Starting from any negative value of $P_{0}<0$, this difference equation can be used to compute a value for $P_{1}$. Then that value for $P_{1}$ can be used to compute $P_{2}$, and so on for all $t=0,1,2,...$. Since the difference equation is ``asymptotically stable,'' this iterative technique will converge to the same negative value of $P$ that you found using the quadratic formula in part (d) above. Try it out using a calculator or computer and see!
A surprisingly large number of economic problems can be written in this linear-quadratic form once $y_{t}$ and $z_{t}$ are allowed to be vectors of state and control variables; and often, even when the problem is not immediately in the linear-quadratic form, it can be approximated by an ``LQ'' problem by taking a second-order Taylor approximation to a non-quadratic objective function and/or a first-order Taylor approximation to a set of nonlinear constraints. In those cases, $R$, $Q$, $A$, and $B$ become matrices of parameters and extensions of both the quadratic formula and the iterative methods that you used above can be applied to the matrix form of the Riccati equation to solve for $P$. For details, see the book by Lars Hansen and Thomas J. Sargent, titled \emph{Recursive Models of Dynamic Linear Economies}.
{\bfseries 2. Natural Resource Depletion}
The now-quite-familiar natural resource depletion problem can also be solved via dynamic programming using the guess-and-verify method. Let $c_{t}$ denote society's consumption of the exhaustible resource during each period $t=0,1,2,...$, and let $s_{t}$ denote the stock of the resource that remains at the beginning of each period $t=0,1,2,...$. Then the social planner chooses sequences $\{c_{t}\}_{t=0}^{\infty}$ and $\{s_{t}\}_{t=1}^{\infty}$ to maximize a representative consumer's utility function
$$
\sum_{t=0}^{\infty} \beta^{t} \ln(c_{t}),
$$
where the discount factor satisfies $0<\beta<1$, subject to the constraints that the initial stock $s_{0}$ is given and that the stock evolves according to
$$
s_{t}-c_{t} \geq s_{t+1}
$$
for all $t=0,1,2,...$.
\begin{description}
\item a. Write down the Bellman equation for this problem.
\item b. Now guess that the value function takes the time-invariant form
$$
v(s_{t};t) = v(s_{t}) = E + F \ln(s_{t}),
$$
where $E$ and $F$ are constants to be determined. Using this guess, write down the first-order condition for $c_{t}$ and the envelope condition for $s_{t}$.
\item c. Your first-order and envelope conditions from part (b) above can be combined with the binding resource constraint
$$
s_{t+1} = s_{t} - c_{t}
$$
to form a system of three equations in three unknowns: the unknown constant $F$ and the unknown variables $c_{t}$ and $s_{t}$ that solve the original dynamic optimization problem. Use the equations from this system to solve for the constant $F$ in terms of the model's single parameter $\beta$.
\item d. Now substitute your solution for $F$ back into the remaining optimality conditions to obtain a system of two equations that can be used to construct the optimal sequences $\{c_{t}\}_{t=0}^{\infty}$ and $\{s_{t}\}_{t=1}^{\infty}$ starting from any initial value of $s_{0}$.
\item e. Just of the sake of completeness, go back to the Bellman equation itself and use your results to solve for the constant $E$ in terms of $\beta$ as well.
\end{description}
\end{document}