Overview of Linear Quadratic Control (LQC), algorithms and implementation examples

Overview of linear quadratic control

Linear-quadratic control (LQR) is a branch of control theory that serves as one of the optimal control methods for systems with linear dynamics.LQR is a method for obtaining feedback control laws to optimally control the state of a system, in particular to minimise a quadratic cost function, and the method is performance is used in designing control strategies that minimise the cost function based on the state and control inputs in order to optimise the performance of the system.

The goal of LQR would be to minimise a cost function of the form

\[
J = \int_0^\infty \left( x(t)^T Q x(t) + u(t)^T R u(t) \right) dt
\]

– \( x(t) \): state vector of the system
– \( u(t) \): control input vector
– \( Q \): weight matrix of states (usually a non-negative definite matrix)
– \(R \): weight matrix of control inputs (usually a positive definite matrix)
– \( J \): cost function

Here, the cost function is defined using the product of the state vector \( x(t) \) and the control input vector \( u(t) \), which minimises the control input \( u(t) \).

LQR is applied to linear systems and the state transitions of the system are represented by the following linear ordinary differential equations

\[
\dot{x}(t) = A x(t) + B u(t)
\]

– \( A \): system matrix (state transition matrix)
– \( B \): input matrix (influence of control inputs)

The goal of LQR would be to optimally adjust the state \( x(t) \) of this linear system using the control input \( u(t) \).

The solution of the LQR can be expressed as follows for the optimal control input \( u(t) \)

\[
u(t) = -K x(t)
\]

where \( K \) is the **gain matrix** and is calculated as follows.

\[
K = R^{-1} B^T P
\]

– \( P \) is a positive definite matrix obtained by the **Riccati equation**.
– This gain matrix \( K \) determines the optimal control input \( u(t) \) based on the state \( x(t) \).

As part of the solution of the LQR, the Riccati equation (Riccati equation) needs to be solved. This equation is the key equation required to obtain the optimum control gain \( K \).

\[
A^T P + P A – P B R^{-1} B^T P + Q = 0
\]

The characteristics of the LQR will be as follows

• Optimality: the LQR provides the optimal solution for a given cost function. This would be a way to minimise the weighted sum of the system’s state transitions and control inputs.
• Stability: the control law obtained by LQR produces a stable system. In particular, the gain matrix \(K \), determined by the solution of the Riccati equation, guarantees the stability of the system.
• Feedback control: the LQR is based on state feedback control, which reflects and controls the system state in real time.

Advantages of using LQR include

• Computational efficiency: LQR is analytically solvable and therefore computationally efficient and can be used in real-time systems.
• Simplicity: it provides a very simple and powerful control method for linear systems.
• Scope of application: it is used in a variety of areas such as robotics, aircraft control, vehicle operation and manufacturing systems.

LQR is a very powerful optimal control method that can be applied to many control problems in linear systems.

implementation example

A simple simulation using Python is presented as an example of linear-quadratic control (LQR) implementation. In this example, a simple one-dimensional system is modelled as the control object and the optimal control inputs are calculated using LQR.

Example: simple one-dimensional linear system

Definition of the system: define the system in terms of linear dynamics as follows \[x(t)=Ax(t)+Bu(t)\]

Where the state\(x(t)\) is the position and the control input\(u(t)\) is the force. The objective of the system will be to control the optimising force such that the state is zero.

• The state transition matrix \(A\) is 1 and the input matrix \(B\) is also 1.
• The cost function is a weighted sum over the states and inputs and is defined as \[ J=\displaystyle\int_0^{\infty}(x(t)^2+u(t)^2)dt\]

Implementation code

import numpy as np
import matplotlib.pyplot as plt
from scipy.linalg import solve_continuous_are

# System parameters.
A = np.array([[0]])  # state-transition matrix
B = np.array([[1]])  # input matrix
Q = np.array([[1]])  # state weight matrix
R = np.array([[1]])  # Input weight matrix

# Solving the Riccati equation
P = solve_continuous_are(A, B, Q, R)

# Calculate the optimal gain matrix K
K = np.linalg.inv(R).dot(B.T).dot(P)

# Simulation of systems
T = 10  # Simulation time
dt = 0.01  # time step
time = np.arange(0, T, dt)

# initial state
x = np.array([1])  # initial position
x_history = []

# control loop
for t in time:
    u = -K.dot(x)  # optimum control input
    x = A.dot(x) + B.dot(u)  # state update
    x_history.append(x[0])

# Plotting the results
plt.plot(time, x_history, label="Position (x)")
plt.xlabel("Time [s]")
plt.ylabel("Position")
plt.title("LQR Control Example")
plt.grid()
plt.legend()
plt.show()

Description.

1. System matrix settings: The state transition matrix \(A\) is \([0]\) and the input matrix \(B\) is \([1]\). In this simple system, the time variation of the position \(x\) is affected by the control input \(u\).
2. Solving the Riccati equation: Use the solve_continuous_are function to solve the Riccati equation. Calculate the optimal gain matrix \(K\) using the matrix \(P\) obtained from the Riccati equation.
3. Computing the control inputs: The optimal control input \(u(t)\) is calculated based on the state \(x(t)\) as follows. [u(t)=-Kx(t)], where \(K\) is thethe gain matrix obtained from the Riccati equation.
4. Simulation:The control loop is turned and the state \(x(t)\) is updated. The state is controlled to converge to zero using the optimal control input \(u(t)\).
5. Plotting the results: Finally, the simulation results (changes in the position (x) are plotted to see how the LQR control converges the state to zero.

RESULTS: After running the simulation, it can be seen that the optimal control acts and the state x(t) converges to zero when the initial position is one. This is a result of the optimal control input provided by the LQR trying to minimise the state.

Application examples

Linear quadratic control (LQR) is widely used in various fields, particularly in control systems, to provide optimal performance. Specific applications of LQR are given below.

1. control of automated vehicles

• Application examples (control of vehicle speed and position): the control of automated vehicles requires optimal adjustment of the vehicle’s acceleration, speed and steering angle LQR is used to control the vehicle’s motion and has been applied for the following purposes
  • Vehicle position control: to move the vehicle towards a target position.
  • Speed control: to optimise deceleration and acceleration while maintaining a constant vehicle speed.
  • Steering angle optimisation: adjusts the steering angle to optimise vehicle handling.

LQR is an approach that can efficiently control vehicle movements by adjusting the weights of states (e.g. position, speed, acceleration) and inputs (e.g. acceleration, steering angle).

2. robot arm control

• Application example (precise positioning of robot arms): industrial robot arms are often controlled using LQR when precise positioning is required. In particular, it is applied to the following tasks.
  • Position control: to ensure that the robot arm moves precisely to the target position.
  • Speed control: optimises the speed of the robot so that it operates efficiently while suppressing vibrations.
  • Torque control: minimises the torque applied to the joints of the robot arm to increase energy efficiency.

LQR can be used to optimally control the motion of the robot arm to enable precision work.

3. aircraft flight control

• Application example (aircraft attitude control and stabilisation): LQR plays a very important role in aircraft flight control. In order to maintain aircraft attitude control and flight stability, LQR is used for the following types of control.
  • Attitude control: the aircraft’s pitch, roll and yaw angles are optimally controlled to maintain stable flight.
  • Altitude control: to optimise acceleration and deceleration while maintaining a constant altitude of the aircraft.
  • Flight path optimisation: adjusts the rudder to fly efficiently towards the target point.

LQR models aircraft dynamics and calculates the optimum control inputs to enable the aircraft to fly safely and stably.

4. unmanned aircraft (drone) control

• Application example (flight stability and position control of drones): LQR is often used to control drones and unmanned aerial vehicles (UAVs). Drones are very sensitive aircraft and real-time position and attitude control is important, and LQR is applied in the following applications
  • Position control: to move the drone to a specified position and reach a target point.
  • Attitude control: optimising the roll, pitch and yaw angles of the drone to ensure stable flight.
  • Speed control: adjusts the drone’s speed to maintain optimal flight.

LQR provides optimum control for the drone’s multi-degree-of-freedom movements and ensures stable flight.

5. power system control.

• Application example (voltage and frequency control of power networks): LQR is also used in power systems. In particular, LQR is used to maintain stable operation in the face of dynamic changes in the supply and consumption of electricity, with specific examples of application as follows
  • Voltage control: optimally adjusting the voltage at various points in the electricity network to maintain a stable supply of electricity.
  • Frequency control: to regulate generation and consumption while maintaining a constant frequency in the electricity network.

LQR can be used to improve the stability of the power network and provide optimal control over transient fluctuations.

6. attitude control of autonomous satellites

• Application example (satellite attitude stabilisation): satellites are controlled using LQR to stabilise their attitude in space. By controlling the attitude of the satellite, the directivity of the antenna can be adjusted to optimise communications and earth observation, and the following controls are used
  • Attitude stabilisation: controls the rotation of the satellite in space and maintains its precise orientation.
  • Orbit correction: corrects the satellite’s orbit and operates in accordance with the specified orbit.

LQR can be used to achieve highly accurate attitude control of the satellite by modelling the satellite dynamics and providing optimal control inputs.

reference book

Reference books on linear quadratic control (LQR) are listed below.

1. ‘Optimal Control Theory: An Introduction’ by Donald E. Kirk
– Description: this book is widely used as an introduction to optimal control theory; LQR is one of the fundamental approaches of this theory and the book explains the theory in detail, from its foundations to its implementation. Concrete examples and calculation methods are also provided to deepen the reader’s intuitive understanding of optimal control problems.

2. ‘Linear System Theory’ by Wilson J. Rugh
– Description: a textbook on linear system theory, covering many problems related to the optimal control of linear systems, including LQR. It provides the mathematical foundations of control theory while also explaining how to apply it to practical problems.

3. ‘Modern Control Engineering’ by Katsuhiko Ogata
– Description: This book covers a wide range of topics in control engineering and is useful for learning the theory and implementation of LQR and linear-quadratic control. It will be easy to understand for students and engineers studying control engineering. 4.

4. ‘Optimal Control and Estimation’ by Robert F. Stengel.
– Contents: provides a detailed description of optimal control and estimation theory; together with the theory associated with LQR, it also describes the general solution of optimal control problems, in particular dealing with extensions to non-linear problems.

5. ‘Control System Design: An Introduction to State-Space Methods’ by Bernard Friedland
– Description: provides a basic approach to control system design using state-space methods; LQR is another control method that uses state-space methods, and you can learn how to implement it in detail in this book.

6. ‘Applied Optimal Control: Optimisation, Estimation, and Control’ by A. E. Bryson Jr. and Y-C Ho
– Description: this book covers a wide range of topics from the basics of optimal control to its applications, including the LQR problem, which is explained in detail, as well as the theoretical background and its application examples.

7. ‘Introduction to Linear Control Systems’
– Description: a book for learning the fundamentals of linear control systems, explaining the linear control theory related to LQR and providing a practical approach to system design.