Jekyll2023-09-04T18:37:17+00:00/feed.xmlUgo RosoliaUgo Rosolia is a research scientist at Amazon. My research interests are in the areas of control, optimization, and data-based algorithms.Model Predictive Control Part II: Learning from data2021-08-06T18:30:25+00:002021-08-06T18:30:25+00:00/jekyll/update/2021/08/06/MPC%20Part%20II<!-- Google tag (gtag.js) -->
<script async="" src="https://www.googletagmanager.com/gtag/js?id=UA-121535981-1"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'UA-121535981-1');
</script>
<p style="text-align: justify">In the <a href="/jekyll/update/2021/04/27/MPC-Part-I.html">previous blog post</a>, we discussed the importance of the terminal constraint and the terminal cost for MPC design. In this post, we will see that these terminal components can be constructed from historical data.
<!-- First, we will focus on iterative tasks. Afterward, we will discuss how to extend the methodology for general control tasks. --></p>
<h2 id="the-control-problem">The Control Problem</h2>
<p style="text-align: justify">We will consider a regulation problem, where our goal is to control a drone that can move only along the vertical direction. The objective of the controller is to hover in place at a given height. We assume that the drone dynamics are described by a linear system and that the cost is quadratic, as shown in the following figure.</p>
<p align="center">
<img src="/images/blog/mpc_II/drone_ex.png" width="800" />
</p>
<p style="text-align: justify">We will use the above example as the system’s state is two-dimensional, and therefore a trajectory of the system can be plotted on a two-dimensional plane. In the following figure, on the two-axis we have the states of the system, i.e., the position and the velocity of the drone.
<!-- The goal state is the origin, as the objective of the control task is to hoven in place at a height given by a nearby cliff from which we measure the vertical position. --></p>
<p align="center">
<img src="/images/blog/mpc_II/drone_tr.png" width="800" />
</p>
<p>In what follows, we are going to iteratively perform the task of steering the drone from a starting state to the goal state, and we will leverage historical data to improve the performance of the MPC.</p>
<h2 id="learning-model-predictive-control">Learning Model Predictive Control</h2>
<p style="text-align: justify">As discussed in the <a href="/jekyll/update/2021/04/27/MPC-Part-I.html">previous blog post</a>, the terminal constraint and terminal cost, often referred to as <em>terminal components</em>, should approximate the tail of the cost and constraint beyond the prediction horizon. The design of these terminal components is crucial to guarantee safety and optimality. In particular, when solving the MPC problem we should use <em>i)</em> a terminal constraint that is given by a safe set of states from which the task can be completed, and <em>ii)</em> a terminal cost function that is a value function representing the cost of completing the task from a safe state.</p>
<p align="center">
<img src="/images/blog/mpc_II/LMPC.png" width="800" />
</p>
<p>Next, we assume that a first feasible trajectory that is able to complete the task is available and we discuss how to construct safe set and value function approximations.</p>
<h2 id="building-safe-sets-from-data">Building Safe Sets from Data</h2>
<p style="text-align: justify">We notice that as the system is deterministic, a state visited during a successful iteration is safe. Indeed, if the system is in a state that we have visited during a successful iteration, we can simply follow the successful trajectory to complete the control task (This fact is trivial for deterministic systems, and for uncertain systems we have to be a bit more careful, please refer to [3]-[4] for further details on uncertain systems). Therefore, we can simply define a safe set of states as the union of the states visited during a successful iteration of the control task.</p>
<p align="center">
<img src="/images/blog/mpc_II/SS.png" width="800" />
</p>
<p style="text-align: justify">The safe set \(\mathcal{SS}^1\) can be used as a terminal constraint for our MPC at each time step. In particular, the MPC will plan an open-loop trajectory that steers the system from the current state back to one of the states visited at the previous iteration of the control task. Then, we apply to the system the first control action and the controller plans a new different open-loop trajectory that steers the system back to the safe set. As shown in the following figures, this replanning strategy allows the MPC to steer the drone away from the safe set.</p>
<p align="center">
<img src="/images/blog/mpc_II/t0.png" width="800" />
</p>
<p style="text-align: justify">The above figure shows the optimal planned trajectory at time \(t = 0\).</p>
<p align="center">
<img src="/images/blog/mpc_II/t1.png" width="800" />
</p>
<p style="text-align: justify">The above figure shows the optimal planned trajectory at time \(t = 1\).</p>
<p align="center">
<img src="/images/blog/mpc_II/tf.png" width="800" />
</p>
<p style="text-align: justify">The above figure shows the closed-loop trajectory at the second iteration of the control task. We notice that at the second iteration the controller was able to explore the state space. Therefore, after completion of the second iteration, we can define a bigger safe set which is given by the union of all data points stored during the successful iterations of the control task. In general, we can define the safe set \(\mathcal{SS}^j\) as the union of the states visited across the first successful \(j\) iterations of the control tasks.</p>
<p align="center">
<img src="/images/blog/mpc_II/SS_j.png" width="800" />
</p>
<p style="text-align: justify">However, we notice that the above safe set renders the optimization problem challenging to solve. Indeed, at each time \(t\) the MPC has to plan a trajectory that lands exactly in one of the states that we have visited. Luckily, it turns out that for linear systems also the convex hull of the stored states is a safe set of states from which the task can be completed. Thus, we can define the convex safe set \(\mathcal{CS}^j\) as the convex hull of all stored states up to the \(j\)th iteration of the control task, as shown in the following figure.</p>
<p align="center">
<img src="/images/blog/mpc_II/cvx.png" width="800" />
</p>
<h2 id="building-value-function-approximations-from-data">Building Value Function Approximations from Data</h2>
<p style="text-align: justify">Now let’s see how we can leverage historical data to construct an approximation to the value function. In the following figure, we have two closed-loop trajectories that successfully steered the drone from the starting state to the goal. Let \(x_k^i\) be the state of the system at time \(k\) of iteration \(i\), then we define \(J_k^i\) as the cost of the roll-out from \(x_k^i\). Such roll-out cost can be simply computed by summing up the cost along the closed-loop realized trajectory. Finally, given a data set of costs and states, we define the value function approximation \(V^j\) as the interpolation of the cost over the stored data, as shown in the following figure.</p>
<p align="center">
<img src="/images/blog/mpc_II/valFun.png" width="800" />
</p>
<p style="text-align: justify">Notice that in the MPC optimization problem the value function will be evaluated at the terminal predicted state, which should belong to the safe region. Therefore, we want to construct a terminal cost function that is defined over the convex safe set \(\mathcal{CS}^j\).
<!-- In particular, we would like to interpolate the cost associated with the roll-out over the convex hull of the stored data. -->
For a given state \(x\) in the convex safe set \(\mathcal{CS}^j\), this interpolation can be computed solving a linear program, as shown in the following figure.</p>
<p align="center">
<img src="/images/blog/mpc_II/valFun_I.png" width="800" />
</p>
<p style="text-align: justify">Given the roll-out data, the value function may be approximated also using different strategies.
However, there is a main advantage in using a linear program to interpolate the cost associated with the stored states: it can be shown that for linear systems this approximated value function is an upper-bound on the future cumulated cost, for more details please refer to [1].</p>
<h2 id="computing-the-control-action">Computing the Control Action</h2>
<p style="text-align: justify">We have discussed how to leverage historical data to construct the terminal components. Now let’s see how these quantities can be used to compute control actions. Finding an explicit expression for the safe set and value function approximation may be challenging and computationally expensive. However, it turns out that an explicit expression is not needed and we can defined an optimization problem that simultaneously computes the terminal components and the optimal control action. In particular, in the optimization problem we can use a sequence of multipliers \(\lambda_k^i\) that are associated with each data tuple \((x_k^i, J_k^i)\) and are used to represent the safe set \(\mathcal{SS}^{j-1}\) and the value function approximation \(V^{j-1}\), as shown in the following figure.</p>
<p align="center">
<img src="/images/blog/mpc_II/LMPC_opt.png" width="800" />
</p>
<p style="text-align: justify">Finally, it is important to underline that for linear systems the above strategy is guaranteed to converge to global optimality when the system dynamics are linear, the constraints polytopic, the cost convex, and a mild technical condition is satisfied [2].</p>
<h2 id="example-and-code">Example and Code</h2>
<p>In this <a href="https://github.com/urosolia/LMPC/tree/master/LinearLMPC">GitHub repo</a>, we implemented the learning MPC (LMPC) strategy from [1] and [2] to solve the following constrained LQR problem:</p>
<p align="center">
<img src="/images/blog/mpc_II/CLQR.png" width="500" />
</p>
<p>The LMPC improves the closed-loop performance until the closed-loop trajectory converges to a steady-state behavior. In this example, the controller iteratively improves the performance until the closed-loop trajectory converges to the unique global optimal solution to the above infinite time constrained LQR problem. For more details please refer to [1] and [2].</p>
<p align="center">
<img src="/images/blog/mpc_II/closed-loop.png" width="350" />
<img src="/images/blog/mpc_II/costImprovement.png" width="350" />
</p>
<h2 id="references-and-code">References and code</h2>
<p>[1] “Learning Model Predictive Control for Iterative Tasks: A Computationally Efficient Approach for Linear System”, <em>U. Rosolia and F. Borrelli</em>, IFAC World Congress, 2017</p>
<p>[2] “On the Optimality and Convergence Properties of the Learning Model Predictive Controller”, <em>U. Rosolia, Y. Lian, E. T. Maddalena, G. Ferrari-Trecate, and Colin N. Jones</em>. To appear on IEEE Transaction on Automatic Control.</p>
<p>[3] “Robust learning model predictive control for linear systems performing iterative tasks”, <em>U. Rosolia, X. Zhang and F. Borrelli</em>, IEEE Transaction on Automatic Control (2021)</p>
<p>[4] “Sample-Based Learning Model Predictive Control for Linear Uncertain Systems”, <em>U. Rosolia and F. Borrelli</em>, Conference of Decision and Control (CDC), 2019.</p>
<p><strong>LMPC Code available on <a href="https://github.com/urosolia/LMPC/tree/master/LinearLMPC">GitHub</a></strong></p>Model Predictive Control Part I: Sufficient conditions for safe policy synthesis2021-04-27T18:30:25+00:002021-04-27T18:30:25+00:00/jekyll/update/2021/04/27/MPC%20Part%20I<!-- Google tag (gtag.js) -->
<script async="" src="https://www.googletagmanager.com/gtag/js?id=UA-121535981-1"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'UA-121535981-1');
</script>
<p style="text-align: justify">Model Predictive Control (MPC) is an established control methodology which systematically uses forecasts to compute control actions.
This control methodology is ubiquitous in industry, with applications ranging from autonomous driving to large scale interconnected power systems.
MPC owes its popularity to the simplicity of the control design, which allows the controller to naturally account for both state and input constraints.
In this post, we are going to discuss sufficient conditions for designing safe MPC policies, which guarantee constraint satifcation and closed-loop stability.</p>
<h2 id="the-control-problem">The Control Problem</h2>
<p style="text-align: justify">First let’s introduce the control problem that we aim to solve. Informally, the goal of control synthesis is to design a policy which guarantees safety and optimal performance for the closed-loop system. These objectives can be often described by a dynamic optimization problem. For instance, consider a dynamical system governed by the update equation \(x_{t + 1} = f(x_t, u_t)\), where the control input \(u_t \in \mathbb{R}^{d}\) and the system state \(x_t \in \mathbb{R}^{n}\).
For a control task starting at \(x(0)\), the control policy should apply to the system the optimal sequence of inputs \(U_T^* = \{u_0^*, u_1^*,...\}\)
solving the following infinite time optimal control problem:</p>
\[\begin{align}
J^*(x(0)) = \min_{u_{0},u_{1},\ldots} \qquad & \sum_{t=0}^{\infty} h(x_{t},u_{t}) \label{eq:FTOCP_RunningCost}\\
\textrm{subject to:} \quad & x_{t+1}=f(x_{t},u_{t}), \forall t\in \{0,1,\ldots\}\\
&x_{t} \in \mathcal{X},u_{t}\in \mathcal{U}, \forall t\in \{0,1,\ldots\} \label{eq:FTOCP_TaskConstr}\\
& x_0=x(0).
\end{align}\]
<p style="text-align: justify">In the above dynamic optimization problem, safety is naturally encoded through the sets \(\mathcal{X}\) and \(\mathcal{U}\), which represent state and input constraints. Finally, the performance of the closed-loop system is quantified by the running cost \(h(\cdot, \cdot)\) which we would like to minimize.</p>
<h2 id="predictive-control">Predictive Control</h2>
<p style="text-align: justify">For a particular task starting from \(x(0)\), one may be tempted to solve Problem \(J^*(\cdot)\) and apply the optimal sequence of control actions to the system. There are two difficulties associated with this idea.
First, the infinite (or long) duration of the task can easily render the solution to Problem \(J^*(\cdot)\) hard to compute.
Second, the model \(f(\cdot)\) used to predict the evolution of the real system can be inaccurate, and even small prediction errors which cumulate at each time step can compromise the success and/or optimality of the closed-loop behavior. To alleviate both issues, it is common practice to \((i)\) solve a control problem over a horizon \(N\) shorter than the task duration and \((ii)\) continuously measure the state of the system, say once every time step, and then recompute new control sequences with updated information from the environment.
Commonly, the procedure we have described above is referred to as \(Model\) \(Predictive\) \(Control\) \((MPC)\). At the generic time \(t\), an MPC policy solves the following problem:</p>
\[\begin{align}
J^{MPC}(x(t)) = \min_{u_{0},\ldots,u_{N-1}} \qquad & \sum_{k=t}^{t+N-1} h(x_{k},u_{k}) + V(x_{t+N}) \\
\textrm{subject to:} ~~\quad & x_{k+1}=f(x_{k},u_{k}), \forall k \in \{ t,\ldots, t+N-1 \} \\
&x_{k} \in \mathcal{X}, u_{k}\in\mathcal{U}, \forall k \in \{ t,\ldots, t+N-1 \} \\
&x_N \in \mathcal{X}_N, \\
& x_0=x(t).
\end{align}\]
<p>where \(x(t)\) is the measured state at time \(t\).</p>
<p style="text-align: justify">Let \(U_N^* =\{u^*_{0}(x(t)),\ldots,u^*_{N-1}(x(t))\}\) be the optimal solution to the above problem at time \(t\).
Then, the first element of \(U_N^*\) is applied to the system and the resulting MPC policy is:</p>
\[\begin{align}
\pi^{MPC}\big(x(t)\big)=u^*_{0}(x(t)).
\end{align}\]
<p style="text-align: justify">In the above equation we use \(u^*_{i}(x(t))\) to emphasize that the optimal solution depends on the current state \(x(t)\). Later on, whenever obvious, the simpler notation \(u_i^*\) will be used.
At the next time step \(t+1\), the optimization problem \(J^{MPC}(\cdot)\) is solved again based on the new state \(x(t+1)\), yielding a \(moving\) or \(receding\) \(horizon\) control strategy. We point out that it is important to distinguish between the real state \(x(t)\) and input \(u(t)\) of the system at time \(t\), and the predicted states \(x_t\) and inputs \(u_t\) in the optimization problem.</p>
<!-- Indeed, often a more complex notation is used, where one differentiates between the state $$x_{k|t}$$ at time $$k$$ -->
<!-- predicted at time $$t$$, and the state $$x_{k|t+1}$$ at time $$k$$ predicted at time $$t+1$$. -->
<p style="text-align: justify">Compare the MPC problem \(J^{MPC}(\cdot)\) with the original control problem \(J^*(\cdot)\) that we want to solve. The MPC problem \(J^{MPC}(\cdot)\) is solved over a shorter horizon \(N\), and it uses a terminal cost \(V(\cdot)\) and terminal constraint set \(\mathcal{X}_N\) to “approximate” cost and constraints beyond the perdition horizon, i.e., from time \(t+N\) to completion of the control task.
The choice of the terminal cost \(V(\cdot)\) and terminal constraint \(\mathcal{X}_N\), which are often referred to as \(terminal\) \(components\), are critical in MPC design.</p>
<h2 id="the-importance-of-the-terminal-components">The Importance of the Terminal Components</h2>
<p style="text-align: justify">It is clear that the prediction model plays a crucial role in determining the success of MPC policies. If the prediction model is inaccurate, then the closed-loop system will deviate from the MPC planned trajectory. This deviation may result in poor closed-loop performance and safety constraint violation.
It is less obvious that, when a perfect system model is used to design an MPC policy, the wrong choice of terminal constraint set and terminal cost function may cause undesirable closed-loop behavior.</p>
<p style="text-align: justify">Recall that the MPC problem \(J^{MPC}(\cdot)\) plans the system’s trajectory over a horizon of length \(N\), which is usually much smaller than the task duration.
With short horizon \(N\) the controller only takes shortsighted control actions, which may be unsafe or may result in poor closed-loop performance.
For instance, in autonomous racing a predictive controller that plans the vehicle’s trajectory over a short horizon without taking into account an upcoming curve, may accelerate to the point that safe turning would be infeasible. In this example, shortsighted control actions would force the closed-loop system to violate safety constraints.
Furthermore, shortsighted control actions may lead to poor closed-loop performance. Consider an autonomous agent trying to escape from a maze along the shortest path, a predictive controller may take a sub-optimal decision, if the short prediction horizon does not allow the controller to plan a trajectory which reaches the exit.</p>
<p style="text-align: justify">Next, we illustrate on a numerical example that shortsighted control actions result in unsafe or poor closed-loop behaviors. We consider the drone hovering problem, where our goal is to control a drone that can move only along the vertical axis. In particular, the objective of the controller is to steer the drone at the same height of a nearby cliff shown in the following figure.</p>
<p align="center">
<img src="/images/blog/mpc_I/drone.png" width="300" />
</p>
<p>In this example, the system dynamics can be approximated by the following double integrator system</p>
\[\begin{align}
x_{t+1} =
\bigg[ \begin{matrix}
1 & 1 \\
0 & 1
\end{matrix} \bigg]
x_t + \bigg[\begin{matrix}
0 \\ 1
\end{matrix} \bigg] u_t
\end{align}\]
<p style="text-align: justify">where the state \(x_t\) collects the position of the drone along the vertical axis \(p_t\) and the velocity \(v_t\), i.e., \(x_t = [p_t, v_t]^\top\). The input \(u_t\) represents the acceleration command that may be used to change the drone’s velocity at the next time step.
The above system is subject to the following state and input constraints</p>
<div style="text-align: justify">\[x_t \in \mathcal{X} = \{ x \in \mathbb{R}^2 : ||x||_\infty \leq 5 \} \text{ and } u_t \in \mathcal{U} = \{ u \in \mathbb{R} : ||u||_\infty \leq 0.5 \}, \forall t \geq 0.\]
</div>
<p style="text-align: justify">In order to regulate the system to the origin, we implemented the MPC policy without a terminal cost and constraint, with \(Q = I\), \(R = 100\) and for different horizon lengths \(N \in \{3,4,5\}\).</p>
<p align="center">
<img src="/images/blog/mpc_I/lossOfFeasibility.png" width="450" />
</p>
<p style="text-align: justify">The above figure shows the comparison between the optimal trajectory and the closed-loop trajectories associated with MPC policies designed with \(N \in \{3, 4, 5\}\) and no terminal components. We notice that for \(N=3\) and \(N=4\), the MPC is not able to regulate the system to the origin. In particular, shortsighted control actions steer the drone to a state from which it is impossible remain within the state constraint set \(\mathcal{X}\), i.e., the drone is going too fast towards the floor and collision is inevitable. For this reason, the MPC problem \(J^{MPC}(\cdot)\) becomes unfeasible and the controller fails to regulate the system to the origin. On the other hand, for \(N=5\) the MPC is able to complete the regulation task, but the closed-loop behavior is suboptimal.</p>
<h2 id="sufficient-conditions-for-safety-and-stability">Sufficient Conditions for Safety and Stability</h2>
<p style="text-align: justify">To prevent failure caused by shortsighted control actions, we need to appropriately design the terminal constraint set \(\mathcal{X}_N\) and the terminal cost function \(V(\cdot)\). In particular, we should design a terminal constraint set \(\mathcal{X}_N\) which is a \(control\) \(invariant\), i.e.,</p>
\[\forall x \in \mathcal{X}_N \subseteq \mathcal{X}, \exists u \in \mathcal{U} \text{ such that } f(x,u) \in \mathcal{X}_N \subseteq \mathcal{X}.\]
<p style="text-align: justify">The above condition guarantees that the controller does not take shortsighted control actions, which result in safety constraint violations due to inertia and/or limited actuation. In particular, given the optimal predicted input sequence \([u_0^*, u_1^*, \ldots, u_{N-1}^*]\) and associated trajectory \([x_0^*, x_1^*, \ldots, x_N^*]\) with \(x_N^* \in \mathcal{X}_N\), invariance of the terminal constraint set guarantees that there exists a control action \(u \in \mathcal{U}\) such that the following shifted input sequence \([u_1^*, \ldots, u_{N-1}^*, u]\) is feasible for the MPC problem \(J^{MPC}(x(1))\) at the next time step \(t= 1\).</p>
<p style="text-align: justify">The terminal set ensures that the MPC policy is safe. However, it is not sufficient to guarantee stability of the closed-loop system. Such property may be guaranteed using a terminal cost function which is a \(control\) \(Lyapunov\) \(function\) over the terminal set \(\mathcal{X}_N\), i.e.,</p>
\[\forall x\in \mathcal{X}_N, \exists u \in \mathcal{U} \text{ such that } V(f(x,u)) - V( x ) \leq - h(x,u).\]
<p style="text-align: justify">The above condition guarantees that there exists a feasible control action \(u \in \mathcal{U}\) such that the terminal cost function is decreasing along a trajectory of the closed-loop system. Furthermore, this decreasing property guarantees that the MPC open-loop cost \(J^{MPC}(\cdot)\) is a Lyapunov function for the origin of closed-loop system, i.e.,</p>
\[\begin{align}
J^{MPC}(x(t+1)) - J^{MPC}(x(t)) \leq - h( x(t), u(t)), \forall t \geq 0,
\end{align}.\]
<p style="text-align: justify">\(J^{MPC}(0)=0\) and \(J^{MPC}(x)\) is radially unbounded. Note that computing a control invariant set and a control Lyapunov function maybe hard also for deterministic linear constrained systems. In practice, these terminal components are computed on a small subset of the state space and they affect the size of the region of attraction associated with the MPC, which is given by the \(N\)-step backward reachable set from the terminal constraint set \(\mathcal{X}_N\).</p>
<h2 id="optimality-gap">Optimality Gap</h2>
<p style="text-align: justify">The sufficient conditions that we discussed so far guarantee closed-loop constraint satisfaction and stability. What about optimality? Unfortunately, these conditions are not enough to guarantee optimality of the closed-loop system. In particular, the optimality gap is a function of \((i)\) how well the terminal cost approximates the optimal value function \(J^*(\cdot)\) and \((ii)\) the size of the terminal constraints set, which ideally should be the \(maximal\) \(stabilizable\) \(set\) \(\mathcal{X}_\infty\), i.e., the largest set of states from which the control task can be completed and \((iii)\) the length of the prediction horizon.</p>
<p style="text-align: justify">To illustrate how the terminal set and terminal cost affect the suboptimality gap we rewrite the optimal value function from the original infinite horizon optimal control problem \(J^*\) described in the <em>Control Problem</em> section as</p>
\[\begin{align}
J^*(x(0)) = \min_{u_{0},\ldots, u_{N-1} } \qquad & \sum_{t=0}^{N-1} h(x_{t},u_{t}) + J^*(x_N)\\
\textrm{subject to:} \quad & x_{t+1}=f(x_{t},u_{t}), \forall t\in \{0,\ldots, N-1\}\\
&x_{t} \in \mathcal{X},u_{t}\in \mathcal{U}, \forall t\in \{0,\ldots,N-1\} \\
& x_N \in \mathcal{X}_\infty \\
& x_0=x(0).
\end{align}\]
<p style="text-align: justify">Let \([x_0^*, \ldots, x_N^*]\) and \([u_0^*,\ldots, u^*_{N-1}]\) be the optimal solution to the above problem. Now let’s assume that \([x_0^*, \ldots, x_N^*]\) and \([u_0^*,\ldots, u^*_{N-1}]\) is a feasible solution to the MPC problem \(J^{MPC}(x(0))\), then we have that the associated cost is</p>
\[\begin{align}
\bar J^{bound}(x(0)) = \sum_{t=0}^{N-1} h(x_{t}^*,u_{t}^*) + V(x_N^*).
\end{align}\]
<p style="text-align: justify">Notice that the above cumulative cost is an upper bound on the MPC open-loop cost \(J^{MPC}(x(0))\) and, therefore, we have that</p>
\[\begin{align}
J^{MPC}(x(0)) - J^*(x(0)) \leq \bar J^{bound}(x(0)) - J^*(x(0)) = V(x_N^*) - J^*(x_N^*).
\end{align}\]
<p style="text-align: justify">Basically, we have that the performance of the MPC policy improves as the terminal constraint \(\mathcal{X}_N\) better approximates the maximal stabilizable set \(\mathcal{X}_\infty\) and the terminal control Lyapunov function \(V(\cdot)\) better approximates the optimal cost function \(J^*(\cdot)\). However computing \(\mathcal{X}_\infty\) and \(J^*(\cdot)\) is as complex as solving the original control problem. In future blog posts, we will discuss how the terminal MPC components can be iteratively synthesized to improve the performance of the MPC.</p>