Addressing target loss and actuator saturation in visual servoing of multirotors: A nonrecursive augmented dynamics control approach

Archit Krishna Kamath; Mir Feroskhan

doi:10.1515/nleng-2024-0018

Artikel Open Access

Addressing target loss and actuator saturation in visual servoing of multirotors: A nonrecursive augmented dynamics control approach

Archit Krishna Kamath und Mir Feroskhan

Veröffentlicht/Copyright: 10. August 2024

Veröffentlicht von

Veröffentlichen auch Sie bei De Gruyter Brill

Manuskript einreichen Informationen für Autor*innen

Aus der Zeitschrift Nonlinear Engineering Band 13 Heft 1

Abstract

Traditional position and image-based visual servoing techniques often pose challenges in terms of target loss and actuator saturation. These challenges arise due to the requirement of calculating inverse Jacobians to determine robot motions and the susceptibility of these methods to image noise. To address the aforementioned challenges, this article presents a nonrecursive augmented dynamics control (NRADC) that establishes a direct correlation between variations in image pixels and the thrust and torque commands of a multirotor. By incorporating pixel variations into the dynamics of the multirotor, this approach enables the utilization of state constraints to address the issue of target loss. In addition, the correlation between image pixels and control commands allows for the integration of image noise with system uncertainties. Consequently, a single controller can be designed to simultaneously handle both aspects while considering input constraints, effectively addressing the problem of actuator saturation. Notably, unlike conventional control approaches that handle input and state constraints, the proposed NRADC approach is nonrecursive, making it well suited for implementation on systems with limited on-board computational resources, such as multirotors. The effectiveness of the proposed method is demonstrated through experimental validations, including a performance comparison with a nonrecursive controller from recent literature.

Keywords: nonrecursive control; augmented dynamics; visual servoing; multirotor; state constraints; input constraints; target loss; actuator saturation

1 Introduction

With the exponential advancement in integrated circuits and manufacturing technology, minicomputers have evolved to efficiently pack increased computing resources while conserving energy. This progress enables the execution of intricate image processing tasks on hand-held devices, eliminating the need for the large GPU setups that were prevalent in the past. Consequently, researchers globally have directed their efforts towards developing efficient visual servoing algorithms capable of handling complex tasks, including object recognition, object tracking, pick-and-place maneuvers, and precision docking applications. The burgeoning trends in the visual servoing domain hold significant promise, particularly for multirotor unmanned aerial systems. Their unique capabilities, such as vertical take-off and landing, precise hovering, and agile maneuvering, position them as ideal candidates for visually guided tasks. Applications span diverse fields, encompassing search and rescue operations, drug delivery, condition monitoring, and reporting systems [1–3]. Existing position and image-based visual servoing algorithms rely on the computation of inverse Jacobians to determine the multirotor’s motion, making them computationally expensive and highly susceptible to image noise [4–6]. Moreover, these algorithms do not directly tackle the issues of target loss and actuator saturation.

Recent research has explored the combination of conventional visual servoing with machine learning algorithms to tackle target loss [5,7]. However, these approaches are recursive in nature and primarily applicable when the object of interest is already outside the camera’s line of sight during tracking. They do not actively contribute to keeping the object within the camera’s field of view. Furthermore, in the existing literature, actuator saturation is treated separately from its application to visual servoing, resulting in the need for two distinct solutions and increasing the computational burden on the multirotor’s onboard computer. To overcome these limitations, this article introduces a nonrecursive augmented dynamics control (NRADC) approach for visual servoing. This approach incorporates pixel variations into the multirotor’s dynamics, establishing a direct relationship between pixel variation and the multirotor’s thrust and torque commands. This correlation enables the modelling of target loss as a state constraint problem and the addressing of actuator saturation as an input constraint control problem. However, an additional challenge remains: the development of a nonrecursive robust controller to complement the augmented dynamics, capable of handling both input and state constraint issues effectively.

Previous studies have addressed input constraints using anti-windup controllers, but these approaches neglect state constraints and are vulnerable to system uncertainties and disturbances [8,9]. Model predictive control (MPC) strategies have been proposed to handle both input and state constraints, but their recursive nature makes them computationally expensive and lacking in robustness [10]. To enhance robustness, some studies propose combining an integral sliding mode controller with a linear MPC algorithm or utilizing a receding horizon-based discrete sliding mode algorithm [11,12]. However, these approaches remain recursive.

In the domain of sliding mode control, a second-order sliding mode controller has been demonstrated to be effective in handling systems with output constraints [13]. This approach incorporates a barrier Lyapunov function and a power integrator term to ensure the system’s output stays within predefined boundaries. However, it does not consider input constraints. Conversely, a chattering-free sliding mode control law based on unidirectional auxiliary surfaces has been introduced to handle state constraints [14]. Nevertheless, this work does not account for input constraints and primarily focuses on limiting the rate of change of the state rather than the state itself.

The limitations mentioned earlier are addressed in a study [15] that deals with a second-order system subject to both input and state constraints. The proposed algorithm divides the operational region into two zones with fixed control effort values corresponding to the maximum allowable input constraint. However, this approach results in significant chattering in the control channel, leading to frequent actuator saturation and instability in systems utilizing camera-based indoor localization. An extension of this algorithm to higher-order systems is presented in [16], where a higher-order sliding mode controller is proposed to mitigate chattering. Nonetheless, the inclusion of integrator chains makes the system sluggish and susceptible to integral windup.

Based on the findings of Rubagotti and Ferrara [15] and Incremona et al. [16], the proposed NRADC approach is inspired. This approach incorporates a nonrecursive terminal sliding mode controller with a power rate reaching law to address systems with input and state constraints [17,18]. The main contributions of this study are as follows:

Development of an augmented dynamics multirotor model that integrates pixel variations into the dynamics of the multirotor. This model establishes a correlation between pixel accelerations and the thrust and torque commands of the multirotor.
Formulation of a nonrecursive robust control law based on a terminal sliding mode controller with a power rate reaching law. This controller is designed to handle systems with both input and state constraints.
Development of a controller gain tuning algorithm to ensure the convergence of error trajectories within a desired boundary of the sliding manifold while strictly adhering to input and state constraints.
Real-time validation of the proposed NRADC approach in preventing target loss and actuator saturation in a target tracking task.

This article is organized as follows: Sections 2 and 3 present the augmented dynamics visual servoing model and outline the problem formulation. Controller design and implementation procedures are addressed in Sections 4 and 5, respectively. Sections 6 and 7 provide details on numerical simulations and experimental validations, respectively followed by concluding remarks in Section 8.

2 Mathematical model

This section presents the details pertaining to the development of the augmented multirotor dynamics for a 6DoF multirotor by extending the idea presented by Kamath et al. [19].

2.1 Multirotor’s dynamical model

The 6-degree-of-freedom multirotor model necessitates 12 states to depict its motion in three-dimensional space. The equation of motion for the multirotor can be expressed through state space representation, as depicted below [20].

(1) x ¨ y ¨ z ¨ ϕ ¨ θ ¨ ψ ¨ = 0 0 − g a 1 θ ˙ ψ ˙ a 2 ψ ˙ ϕ ˙ a 3 ϕ ˙ θ ˙ + 1 m 0 0 0 0 0 0 1 m 0 0 0 0 0 0 1 m 0 0 0 0 0 0 1 J x 0 0 0 0 0 0 1 J y 0 0 0 0 0 0 1 J z u x u y u z u ϕ u θ u ψ ⇒ η ¨ o η ¨ in = f ¯ o f ¯ in + J o 0 0 J in u ¯ o u ¯ i + D o D in ,

where:

x , y , and z represent the position and altitude of the multirotor in the inertial frame.
ϕ , θ , and ψ represent the roll, pitch, and yaw angles of the multirotor with respect to the inertial frame satisfying the conditions − π 2 < ϕ < π 2 , − π 2 < θ < π 2 , and − π < ψ < π .
u x , u y , and u z represent the axillary control inputs of the multirotor and are given as follows:
(2) u x = u T ( c ϕ s θ c ψ + s ϕ s ψ ) u y = u T ( c ϕ s θ s ψ − s ϕ c ψ ) u z = u T ( c ϕ c θ ) .
c i and s i are a short hand notation for cos ( i ) and sin ( i ) , respectively ∀ i = ϕ , θ , and ψ .
u T , u ϕ , u θ , and u ψ represent the thrust, roll, pitch, and yaw control inputs, respectively, of the multirotor.
J x , J y , J z , and m represent inertial constants about the x , y , and z -axis and mass of the multirotor, respectively.
g represents the acceleration due to gravity.
a 1 , a 2 , and a 3 are terms that are defined as: a 1 = J y − J z J x , a 2 = J z − J x J y , and a 3 = J x − J y J z .
η o ∈ R 3 and η in ∈ R 3 represent the outer and inner loop state variables, respectively, of the multirotor and are defined as η o = [ x y z ] T and η in = [ ϕ θ ψ ] T .
f ¯ o ∈ R 3 and f ¯ in ∈ R 3 are functions that represent the nonlinear dynamics of the multirotor’s outer and inner loop, respectively, and are defined as f ¯ o = [ 0 0 − g ] T and f ¯ in = [ a 1 θ ˙ ψ ˙ a 2 ψ ˙ ϕ ˙ a 3 ϕ ˙ θ ˙ ] T .
J o ∈ R 3 × 3 and J in ∈ R 3 × 3 represent the Jacobian that relates the multirotor inputs to its states in the outer and inner loop respectively. These are given as J o = diag [ 1 m 1 m 1 m ] and J i = diag [ 1 J x 1 J y 1 J z ] .
u ¯ o ∈ R 3 represents the auxiliary input vector to the outer loop of the multirotor model and is defined as u ¯ o = [ u x u y u z ] T .
u ¯ i ∈ R 3 represents the input vector of the multirotor’s inner loop and is given as u ¯ in = [ u ϕ u θ u ψ ] T .
D o ∈ R 3 and D i ∈ R 3 represent the bounded disturbance in the outer loop and inner loop state variables, respectively.

To obtain the thrust input of the multirotor and the inner loop desired values of the roll and pitch angle, (2) is used as follows:

(3) u T = u x 2 + u y 2 + u z 2 θ d = tan − 1 u x c ψ d + u y s ψ d u z ϕ d = tan − 1 u x s ψ d c θ d − u y c ψ d c θ d u z .

2.2 Acceleration image dynamics

This subsection presents the acceleration image dynamics. To understand this, consider a 3D point P ∈ R 3 in space represented by P = [ P x P y P z ] . The 2D coordinate of this point in the camera frame is obtained using the following relationship [21]:

(4) p = u v T = f P z P x P y T ,

where f ∈ R represents the focal length of the monocular camera, obtained using the process of calibration, and p ∈ R 2 = u v T represents the coordinate of the transformed 3D point, P , in the camera frame. p is also termed a point of interest (POI).

In a situation where the 3D point, P , starts to move in the 3D space, a corresponding change is observed in the 2D camera frame. The relationship governing this motion in the 2D camera frame is given by:

(5) p ˙ = − f P z 0 u P z u v f − f + u 2 f v 0 − f P z v P z f + v 2 f − u v f − u V ¯ ,

where p ˙ ∈ R 2 represents the velocity of the point p , L s ν ∈ R 2 × 3 and L s ω ∈ R 2 × 3 represent the linear and angular velocity component of the camera interaction matrix for the point p and V ¯ ∈ R 6 = ν ω T with ν ∈ R 3 and ω ∈ R 3 representing the 3D linear and angular velocities of the point p , respectively. To obtain the augmented dynamical multirotor model, consider extending the relationship presented in (5) for the motion of “ n ” points of interest in the 3D plane:

(6) r ˙ = p ˙ 1 p ˙ 2 ⋮ p ˙ n = − f P z 1 0 u 1 P z 1 u 1 v 1 f − f + u 1 2 f v 1 0 − f P z 1 v 1 P z 1 f + v 1 2 f − u 1 v 1 f − u 1 − f P z 2 0 u 2 P z 2 u 2 v 2 f − f + u 2 2 f v 2 0 − f P z 2 v 2 P z 2 f + v 2 2 f − u 2 v 2 f − u 2 ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ − f P z n 0 u n P z n u n v n f − f + u n 2 f v n 0 − f P z n v n P z n f + v n 2 f − u n v n f − u n V ¯ ,

where u i , v i , and P z i represent the pixel address and the depth of the i th POI, respectively, and r ˙ ∈ R 2 n represents the velocity of the “ n ” POIs. (6) can be rewritten as follows:

(7) r ˙ = L s ν 1 ∣ L s ω 1 L s ν 2 ∣ L s ω 2 ⋮ L s ν n ∣ L s ω n V ¯ ,

where r ˙ ∈ R 2 n represents the velocity of the POIs and L s ν i ∈ R 2 × 3 and L s ω i ∈ R 2 × 3 represent the linear and angular velocity component of the camera interaction matrix for the i th POI, respectively. Since acceleration dynamics are required for augmentation, (7) needs to be differentiated with respect to time. This is as follows:

(8) r ¨ = L s ν 1 ∣ L s ω 1 L s ν 2 ∣ L s ω 2 ⋮ L s ν n ∣ L s ω n A ¯ + L μ 1 L μ 2 ⋮ L μ n + D r 1 D r 2 ⋮ D r n ,

where

A ¯ ∈ R 6 = a α T with a ∈ R 3 and α ∈ R 3 representing the 3D linear and angular accelerations of the POIs, respectively.
L μ i ∈ R 2 is a vector, which is given as V ¯ T Δ i V ¯ V ¯ T Ξ i V ¯ T . Δ i ∈ R 6 × 6 and Ξ i ∈ R 6 × 6 are obtained as shown in (9).
D r i ∈ R 2 represents the bounded image disturbance vector associated with the i th POI.

Δ i = 0 0 f P z i 2 − v i P z i 3 u i 2 P z i 0 0 0 0 − u i 2 P z i 0 − f 2 P z i f P z i 2 0 2 u i P z i 2 2 u i v i f P z i − f 2 P z i − 2 u i 2 f P z i v i P z i − v i P z i − u i 2 P z i 2 u i v i f P z i u i + 2 u i v i 2 f 2 − v i 2 − 2 u i 2 v i f 2 f 2 − u i 2 2 f + v i 2 f 3 u i 2 P z i 0 − f 2 P z i − 2 u i 2 f P z i − v i 2 − 2 u i 2 v i f 2 2 u i + 2 u i 3 f 2 − 3 u i v i 2 f 0 − f 2 P z i v i P z i f 2 − u i 2 2 f + v i 2 f − 3 u i v i 2 f − u i

(9) Ξ i = 0 0 0 0 2 v i P z i f 2 P z i 0 0 f P z i 2 − 3 v i 2 P z i u i P z i 0 0 f P z i 2 2 v i P z i 2 f 2 P z i + 2 v i 2 f P z i − u i v i f P z i − u i P z i 0 − 3 v i 2 P z i − f 2 P z i + 2 v i 2 f P z i 2 v i + 2 v i 3 f 2 − u i 2 − 2 u i v i 2 f 2 − 3 u i v i 2 f f 2 P z i u i v i − u i v i f P z i − u i 2 − 2 u i v i 2 f 2 v i + 2 u i 2 v i f 2 f 2 + u i 2 f − v i 2 2 f f 2 P z i 0 − u i P z i 3 u i v i 2 f f 2 + u i 2 f − v i 2 2 f − v i

(1) and (8) can be combined to obtain the augmented dynamical model.

2.3 Augmented dynamical multirotor model

In order to derive the augmented dynamical model of the multirotor, the outer loop state variables of the multirotor are employed. This approach is taken to ensure the effective tracking of POIs using the multirotor UAV. Consequently, the dynamics considered for augmentation are extracted from (1) and are presented as follows:

(10) η ¨ o = f ¯ o + J o u ¯ o + D o .

Since pixel acceleration in (8) is a for 2D pixel values and the outer loop dynamics of the multirotor has three variables, the relationship between them will have to be obtained by splitting the outer loop dynamics into two subsections as shown:

(11) η ¨ dep η ¨ pos = f ¯ dep f ¯ pos + J dep 0 0 J pos u ¯ dep u ¯ pos + D dep D pos ,

where the subscripts dep and pos represent the depth and the position subsystem, respectively. For instance, if the camera setup is as depicted in Figure 1, the dep parameter would align with the z axis of the multirotor and the pos parameter would align with the x and y axes of the multirotor.

Figure 1

Multirotor and camera setup.

The augmented dynamical model can be obtained using the pos subsystem alongside (8). To relate the variables of (8) with that of (11), a rotation matrix R cam UAV ∈ R 2 × 2 is considered that relates the multirotor’s body frame and the camera frame. The product R cam UAV η ˙ pos R 2 and R cam UAV η ¨ pos ∈ R 2 are related to V ¯ and A ¯ as follows:

(12) V ¯ = R cam UAV η ˙ pos 0 0 0 0 T and A ¯ = R cam UAV η ¨ pos 0 0 0 0 T .

Substituting (12) in (8) and eliminating the zero columns:

(13) r ¨ = L s ν ′ 1 L s ν ′ 2 ⋮ L s ν ′ n R cam UAV η ¨ pos + L μ ′ 1 L μ ′ 2 ⋮ L μ ′ n + D r 1 D r 2 ⋮ D r n ,

where L s ν ′ i ∈ R 2 × 2 is the reduced interaction matrix with components necessary to enable movements in x and y directions of the camera and L μ ′ i ∈ R 2 × 1 is given as follows:

(14) L μ ′ i = ( R cam UAV η ˙ pos ) T Δ i R cam UAV η ˙ pos ( R cam UAV η ˙ pos ) T Ξ i R cam UAV η ˙ pos . T .

For ease of representation, (13) can be rewritten as follows:

(15) r ¨ = F aug + J aug u ¯ pos + D aug ,

where F aug ∈ R 2 n , J aug ∈ R 2 n × 2 , and D aug ∈ R 2 n (transformed bounded disturbance vector) are functions of r , r ˙ , η pos , and η ˙ pos , respectively. From (11) and (15), the overall outer loop dynamics are given as follows:

(16) η ¨ dep r ¨ = f ¯ dep F aug + J dep 0 2 n × 1 0 J aug u ¯ dep u ¯ pos + D dep D aug .

Therefore, the outer loop control is now split into two subsystems: the depth subsystem and the augmented dynamics position subsystem. The state of the multirotor in each of these subsystems is influenced by the camera’s orientation in relation to the multirotor.

Remark 1

The transformed bounded disturbance vector, i.e., D aug combines the system disturbance D pos and the bounded image noise D r i for each POI being tracked. This merger facilitates in designing of a single controller to tackle both.

Remark 2

The following assumptions are made for this work: The intrinsic parameters of the camera being used are fully known. Uncertainties in the depth estimations and intrinsic parameters are assumed to be minimal. A LiDAR is employed in conjunction with the monocular camera to facilitate the acquisition of depth.

3 Problem formulation

Consider the complete augmented dynamics multirotor model obtained from (1) and (16):

(17) η ¨ dep r ¨ η ¨ in = f ¯ dep F ¯ aug f ¯ in + J dep u ¯ dep G ¯ aug u ¯ pos J in u ¯ in + D dep D ¯ aug D in .

Let the aforementioned dynamics be subjected to a set of state constraints X q and input constraints U given as follows:

(18) X q = η dep - min < η dep < η dep - max r min < r < r max η in - min < η in < η in - max η ˙ dep - min < η ˙ dep < η ˙ dep - max r ˙ min < r ˙ < r ˙ max η ˙ in - min < η ˙ in < η ˙ in - max ,

(19) U = − U ¯ dep ≤ u ¯ dep ≤ U ¯ dep − U ¯ pos ≤ u ¯ pos ≤ U ¯ pos − U ¯ in ≤ u ¯ in ≤ U ¯ in .

Considering the imposed constraints, the following conditions hold: ∣ f ¯ dep ∣ ≤ F dep - max , ∣ F ¯ aug ∣ ≤ F ¯ aug - max , ∣ f ¯ in ∣ ≤ f ¯ in - max , and 0 < G ¯ aug - min ≤ G ¯ aug ≤ G ¯ aug - max . Moreover, according to the dynamical model, J dep ∈ R + , and J in is a positive definite diagonal matrix with constant elements representing the inertia of the multirotor. The objective now is to develop a controller that will help converge the system states in (17) to their desired values, i.e., η dep d , r d and η in d when adhering to the constraints presented in (18) and (19).

Remark 3

The constraint U prevents actuator saturation, while the constraint on r and r ˙ within X q aims to prevent target loss. To achieve this, r is bounded by the camera resolution, ensuring it does not exceed the resolution limit.

4 Control strategy design

Consider a nonlinear system represented by the equation:

(20) x ˙ 1 = x 2 , x ˙ 2 = f ( x 1 , x 2 ) + g ( x 1 , x 2 ) u + d ,

where x 1 ∈ R and x 2 ∈ R represent the system states, u ∈ R represents the system input, f ( x 1 , x 2 ) ∈ R and g ( x 1 , x 2 ) ∈ R are smooth functions that are used to represent the dynamics of the second order system, and d ∈ R represents the bounded external disturbance term. Let the system, represented by (20), be subjected to the state and input constraints as follows:

(21) X = x 1 min < x 1 < x 1 max x 2 min < x 2 < x 2 max − U ≤ u ≤ U .

This effectively implies that the functions f ( x 1 , x 2 ) and g ( x 1 , x 2 ) are bounded and can be represented as ∣ f ( x 1 , x 2 ) ∣ ≤ F and 0 < G 1 ≤ g ( x 1 , x 2 ) ≤ G 2 . Having these bounds also helps in obtaining an essential condition on U . The objective is to design a chattering-free robust control law u that can steer the system states to zero, while ensuring that the state and input constraints ( X ) are satisfied for all t > 0 . To achieve this, consider the control effort u ( t ) as follows:

(22) u = − U S 1 ∪ S 2 ∪ S 3 U S 4 ∪ S 5 ∪ S 6 μ else ,

where μ ∈ R is derived using a terminal sliding mode control strategy with a power rate reaching law. The controller parameters are tuned such that − U ≤ μ ≤ U ∀ t > 0 . The sets S 1 − 6 are defined as follows:

(23) S 1 ≜ [ ( x 1 , x 2 ) : x 2 ≥ x 2 max and x 1 > x 1 min ] S 2 ≜ ( x 1 , x 2 ) : x 1 ≥ − ∣ x 2 ∣ γ λ + x 1 max and x 2 > 0 S 3 ≜ [ ( x 1 , x 2 ) : x 1 ≥ x 1 max and x 2 ≤ 0 ] S 4 ≜ [ ( x 1 , x 2 ) : x 2 ≤ x 2 max and x 1 < x 1 max ] S 5 ≜ ( x 1 , x 2 ) : x 1 ≤ ∣ x 2 ∣ γ λ + x 1 max and x 2 < 0 S 6 ≜ [ ( x 1 , x 2 ) : x 1 ≤ x 1 max and x 2 ≥ 0 ] ,

where γ ∈ R + and λ ∈ R + are surface parameters such that 1 < γ = m ∕ n < 2 and m and n are odd numbers. Figure 2 depicts these sets for certain values of x 1 min , x 1 max , λ , and γ .

$Figure 2 Sets S 1 − 6 {{\mathcal{S}}}_{1-6} for x 1 min = x 1 min = − 1 {x}_{{1}_{\min }}={x}_{{1}_{\min }}=-1 , x 1 max = x 1 max = 1 {x}_{{1}_{\max }}={x}_{{1}_{\max }}=1 , λ = 1 \lambda =1 , and γ = 5 ∕ 3 \gamma =5/3 .$

Figure 2

Sets S 1 − 6 for x 1 min = x 1 min = − 1 , x 1 max = x 1 max = 1 , λ = 1 , and γ = 5 ∕ 3 .

To derive the control effort μ , consider a nonsingular terminal sliding manifold as follows:

(24) S = ∣ x 2 ∣ γ sign ( x 2 ) + λ x 1 .

This nonlinear sliding manifold ensures that the state trajectories converge to zero in finite time t s . The control effort μ can be computed by taking the time derivative of (24) and substituting (20) in it:

(25) μ = − g − 1 f + λ γ ∣ x 2 ∣ 2 − γ sign ( x 2 ) + K ∣ S ∣ α sign ( S ) ,

where K ∈ R + and 0 < α < 1 represent the reaching law parameters. The power rate reaching law helps in converging the trajectories onto the sliding manifold in finite time t r making the total convergence time as t f = t r + t s .

4.1 Stability analysis

The stability analysis begins with establishing that the control effort μ is finite time stable and obtaining expressions for t s and t r . This is then followed by deriving conditions on the controller parameters, λ , K , γ , and α to ensure that μ ∈ ( − U , U ) . The last stage of this analysis will show that the nonlinear system, shown in equation (20), will satisfy the state constraints X under the influence of the controller, proposed in equation (22).

Theorem 1

For the given nonlinear system described in (20), when subjected to the control effort defined in (25), it can be proven that the system will converge to a small neighborhood in the vicinity of the sliding manifold specified by (24) within a finite time interval t f .

Proof

Considering a positive definite Lyapunov candidate function V = ( 1 ∕ 2 ) S 2 . Taking its derivative and substituting equations (20) and (25):

(26) V ˙ = S γ ∣ x 2 ∣ γ − 1 f ( x 1 , x 2 ) − f ( x 1 , x 2 ) + λ γ ∣ x 2 ∣ 2 − γ sign ( x 2 ) + K ∣ S ∣ α sign ( S ) ) + d } + λ x 2 ]

Simplifying the aforementioned equation further, we obtain

(27) V ˙ = − γ ∣ x 2 ∣ γ − 1 [ K ∣ S ∣ α + 1 − S d ] = − γ ∣ x 2 ∣ γ − 1 K − d ∣ S ∣ α ∣ S ∣ α + 1 .

To further simplify the aforementioned inequality, three cases are considered:

x 2 ≠ 0 and d = 0 : It is necessary to have V ˙ < 0 to guarantee system stability. Given that x 2 ≠ 0 and d = 0 , for any positive gain value K , the system will exhibit finite-time stability, causing the system states to converge to the origin within a duration of t f .
x 2 ≠ 0 and d ≠ 0 : Similar to the previous approach, to satisfy V ˙ < 0 , it is necessary to ensure that K is greater than d max ∕ ∣ S min ∣ α . However, it is important to note that when ∣ S min ∣ α = 0 , the value of K becomes infinite ( K > ∞ ). This is not practical, so we select a small value for ∣ S min ∣ and denote it as ε . This ε represents a boundary in the vicinity of the sliding manifold, where the system state trajectories will converge.
x 2 = 0 : To analyze this situation, let’s substitute equation (25) into equation (20). We observe that when x 2 = 0 , x ˙ 2 = − K ∣ S ∣ α sign ( S ) . This implies that when x 2 = 0 , x ˙ 2 is not necessarily zero because S = λ x 1 may not be zero. Consequently, if x ˙ 2 ≠ 0 , then x 2 cannot remain at zero. This means that, for nonzero values of x 1 , the state trajectories will not be able to remain at x 2 = 0 . Therefore, x 2 = 0 is not an attractor, and the influence of x 2 = 0 on inequality (27) can only be considered when both x 1 and x 2 simultaneously reach zero, indicating the convergence of the system states to the origin [22].□

Based on the analysis of the three aforementioned cases, it can be concluded that V ˙ < 0 , ensures system stability. To obtain the convergence time in the sliding phase consider the differential equation in (24):

(28) S = ∣ x 2 ∣ γ sign ( x 2 ) + λ x 1 = 0 .

Solving the aforementioned differential equation, the convergence time can be obtained using the expression [23]:

(29) t s = γ ∣ x 1 ( 0 ) ∣ ( γ − 1 ) ∕ γ ( γ − 1 ) λ 1 ∕ γ .

On similar lines, the power rate reaching law can also prove to be finite-time convergent by substituting S ˙ = − K ∣ S ∣ α sign ( S ) in V ˙ :

(30) V ˙ = S S ˙ = − K ∣ S ∣ α + 1 = − 2 K V α + 1 2 .

Since, 0 < α < 1 , the power α + 1 2 < 1 , which implies that the reaching phase is finite time convergent. The expression for the time of convergence t r can be obtained by solving the differential equation shown in (30):

(31) ∫ V ( 0 ) 0 d V V α + 1 2 = − ∫ 0 t r 2 K d t ⇒ t r = 2 K ∣ V ( 0 ) ∣ 1 − α 2 1 − α .

The following part of this section is dedicated to proving that the control algorithm effectively confines the system states within the desired state constraints.

Theorem 2

When applying the control law u described by (22) to the nonlinear system in (20) subject to the state and input constraints denoted by X , the system’s state trajectories will remain confined to a positively invariant region ℛ ′ defined as ℛ ′ ≜ ℛ \ ⋃ i = 1 6 S i .

Proof

Consider the region ℛ ′ depicted in Figure 2. The boundary of ℛ ′ consists of six segments: A 1 A 2 ¯ , A 2 A 3 ¯ , A 3 A 4 ¯ , A 4 A 5 ¯ , A 5 A 6 ¯ , and A 6 A 1 ¯ . From (24), it can be noted that the first three segments are influenced by the control effort − U , while the remaining segments are influenced by the control effort U . The objective is to examine the initial conditions of the system, denoted as x 0 ≔ ( x 1 ( 0 ) , x 2 ( 0 ) ) , on each of these segments and demonstrate that the state trajectories will consistently point inward toward ℛ ′ under the influence of the proposed controller.

x 0 ∈ A 1 A 2 ¯ and x 0 ∈ A 4 A 5 ¯ :
Noting that points A 1 and A 4 are already covered in segments A 1 A 2 ¯ and A 4 A 5 ¯ , we focus on the proof for segments A 6 A 1 ¯ and A 3 A 4 ¯ . Considering the case when x 0 ∈ A 1 A 2 ¯ . Using (24), the control effort u ( t ) = − U , resulting in the system dynamics transforming to x ˙ 1 = x 2 > 0 and x ˙ 2 = f ( x 1 , x 2 ) − g ( x 1 , x 2 ) U . Since ∣ f ( x 1 , x 2 ) ∣ ≤ F and 0 < G 1 ≤ g ( x 1 , x 2 ) ≤ G 2 , we obtain the following inequality − ( F + G 2 U ) ≤ f − g U ≤ F − G 1 U < 0 . This implies that x ˙ 2 < 0 , causes the vector field to point right and downwards, thus directing the resultant vector field into the PI set ℛ ′ . Once the trajectories enter ℛ ′ , the control effort μ facilitates the convergence of trajectories towards the origin, as demonstrated in the previous proof. A similar situation arises when x 0 ∈ A 4 A 5 ¯ with u ( t ) = U , resulting in the vector field pointing left and upwards.
x 0 ∈ A 2 A 3 ¯ and x 0 ∈ A 5 A 6 ¯ :
Consider the scenario when x 0 ∈ A 2 A 3 ¯ . Here, we have x ˙ 1 = x 2 > 0 and x ˙ 2 = f ( x 1 , x 2 ) − g ( x 1 , x 2 ) U < 0 , as previously shown. However, it is important to note that the vector x 0 always satisfies the equation x 1 = − ∣ x 2 ∣ γ λ + x 1 max . Consequently, the vector field on this segment can, at most, be tangential to the segment and will never point outward from the PI set ℛ ′ . A similar analysis applies when x 0 ∈ A 5 A 6 ¯ .
x 0 ∈ A 3 A 4 ¯ and x 0 ∈ A 6 A 1 ¯ :
In the case of x 0 ∈ A 3 A 4 ¯ , we have x ˙ 1 = x 2 < 0 and x ˙ 2 = f ( x 1 , x 2 ) − g ( x 1 , x 2 ) U < 0 . This indicates that the vector field on this segment directs to the left and downwards, causing the resultant vector to point into the PI set ℛ ′ . Similarly, a comparative analysis can be conducted for the situation when x 0 ∈ A 6 A 1 ¯ .

The vector field will consistently direct inward for any x 0 located on the boundary of ℛ ′ . Consequently, the state trajectories will converge to the origin within a finite time period when subjected to the control algorithm specified in (24).□

4.2 Control parameter tuning

To limit the magnitude of the control effort μ within the input constraint range, i.e., μ ( t ) ∈ ( − U , U ) , it is necessary to determine suitable values for the control parameters λ , K , γ , and α . The selection of α and γ will be based on heuristics to ensure a smooth response. Meanwhile, the values of λ and K will be chosen to adhere to the input constraints. This can be achieved by considering (25) and substituting the extreme values of inputs and system states, as shown:

(32) μ max = G 1 − 1 F + λ γ X 2 2 − γ + S ˙ max < U ,

where X 2 = max [ ∣ x 2 max ∣ , ∣ x 2 min ∣ ] . Considering the reaching law as S ˙ max = K ( S max ) α , and from (24) we have S max = X 2 γ + λ X 1 , where X 1 = max [ ∣ x 1 max ∣ , ∣ x 1 min ∣ ] . Substituting this in (32):

(33) 0 < λ γ X 2 2 − γ + K ( X 2 γ + λ X 1 ) α < G 1 U − F .

The aforementioned inequality is in terms of λ and K . To obtain tuning laws two cases can be considered:

Fixing speed of convergence: If the convergence speed is fixed, the value of λ is also fixed. Consequently, (33) becomes:
(34) K < G 1 U − F − λ γ X 2 2 − γ ( X 2 γ + λ X 1 ) α
Based on the stability analysis presented earlier, the gain K must be chosen as K > d max ∕ ∣ S min ∣ α . By combining these conditions:
(35) d max ∣ S min ∣ α < K < G 1 U − F − λ γ X 2 2 − γ ( X 2 γ + λ X 1 ) α ,
Fixing λ , for a given d max , S min must be selected such that (35) is satisfied. Therefore, fixing the convergence speed compromises the region around the sliding manifold where the state trajectories remain to satisfy (35).
Fixing the region of convergence: If the region of convergence around the sliding manifold is fixed, it implies fixing the value of S min . Given a d max , K can be obtained using the relation K > d max ∕ ∣ S min ∣ α . This value of K can then be used to determine the value of λ from (33) using numerical methods.

In both methods, one of the control law parameters needs to be selected heuristically, while the other is determined based on the input and state constraints. To illustrate the implementation of the proposed controller.

4.3 Implementation example

To demonstrate the effectiveness of the controller, consider the nonlinear second order system:

(36) x ˙ 1 = x 2 x ˙ 2 = sin ( x 1 ) + x 2 + u + d ,

where f ( x 1 , x 2 ) = sin ( x 1 ) + x 2 and g ( x 1 , x 2 ) = 1 . The aforementioned system is subjected to the input and state constraints X , as shown below:

(37) X = − 1 < x 1 < 1 − 1 < x 2 < 1 − 8 ≤ u ≤ 8 .

For X to be satisfied, the control law is selected as per equation (22) with λ = 0.65 , γ = 59 ∕ 53 , and α = 0.85 . Since the system is constrained, one can say that F = 1.8415 and G 1 = G 2 = 1 . Now using the inequality (34), the value of K can be obtained as K < 3.7 . This work considers the convergence of the system states to zero in order to depict the efficacy of the proposed controller. The same is then compared with the controller in [15]. The results for this are as shown in Figures 3 and 4.

$Figure 3 Regulation task results. (a) x 0 = ( 0.9 , 0.9 ) {{\bf{x}}}_{0}=\left(0.9,0.9) , (b) x 0 = ( 0.9 , − 0.9 ) {{\bf{x}}}_{0}=\left(0.9,-0.9) , (c) x 0 = ( − 0.9 , − 0.9 ) {{\bf{x}}}_{0}=\left(-0.9,-0.9) , and (d) x 0 = ( − 0.9 , 0.9 ) {{\bf{x}}}_{0}=\left(-0.9,0.9) .$

Figure 3

Regulation task results. (a) x 0 = ( 0.9 , 0.9 ) , (b) x 0 = ( 0.9 , − 0.9 ) , (c) x 0 = ( − 0.9 , − 0.9 ) , and (d) x 0 = ( − 0.9 , 0.9 ) .

$Figure 4 Control efforts. (a) x 0 = ( 0.9 , 0.9 ) {{\bf{x}}}_{0}=\left(0.9,0.9) , (b) x 0 = ( 0.9 , − 0.9 ) {{\bf{x}}}_{0}=\left(0.9,-0.9) , (c) x 0 = ( − 0.9 , − 0.9 ) {{\bf{x}}}_{0}=\left(-0.9,-0.9) , and (d) x 0 = ( − 0.9 , 0.9 ) {{\bf{x}}}_{0}=\left(-0.9,0.9) .$

Figure 4

Control efforts. (a) x 0 = ( 0.9 , 0.9 ) , (b) x 0 = ( 0.9 , − 0.9 ) , (c) x 0 = ( − 0.9 , − 0.9 ) , and (d) x 0 = ( − 0.9 , 0.9 ) .

Figure 3 represents the phase plot for the system with different initial conditions, x 0 . The goal is to reach the point (0, 0) while ensuring that the input and state constraints, X , are satisfied. Both the controllers are able to reach the desired point; however, Rubagotti and Ferrara [15] produce significant chattering in the state. In addition, in several instances, Rubagotti and Ferrara [15] are unable to maintain the state trajectories within the given constraint. This can be attributed to the fact that the control effort, in the case of [15], can only choose between two extreme values. This may make the system states to move to a value greater than the imposed constraints.

The control effort profile for both the controllers is presented in Figure 4. The controller in [15] produce significant chattering to stabilize the system states to (0, 0) as opposed to the proposed controller. To mathematically assess the performance of the proposed controller with respect to the study by Rubagotti and Ferrara [15], integral square error (ISE), integral absolute error (IAE), integral time absolute error (ITAE), average control effort (ACE), average value of control effort between samples ( Δ u ), and degree of chattering (DOC) are computed for x 0 = ( 0.9 , 0.9 ) and shown in Table 1 [24,25].

Table 1

Performance evaluation

Index	Formula	Proposed	[15]
ISE	∫ 0 T ( ∣ x 1 ∣ + ∣ x 2 ∣ ) 2 d t	0.4374	0.4386
IAE	∫ 0 T ( ∣ x 1 ∣ + ∣ x 2 ∣ ) d t	0.5714	0.8317
ITAE	∫ 0 T t ( ∣ x 1 ∣ + ∣ x 2 ∣ ) d t	0.3461	1.1068
ACE	1 T ∫ 0 T ∣ u ∣ 2 d t	0.6467	2.8261
Δ u	1 N ∑ k = 1 N ∣ u k Δ T ‒ u ( k − 1 ) Δ T ∣ 2	0.3797	14.9131
DOC	Δ u /ACE	0.5871	5.2770

From the entries of Table 1, one can see that the convergence error, to the origin, is smaller in the case of the proposed controller, as opposed to [15]. In addition, the DOC and the average control effort between samples are also lesser in case of the proposed controller. Similar performance indices can be observed for different values of x 0 . This effectively established that the proposed controller performs more effectively as compared to that in [15]. This algorithm can now be extended to control the augmented dynamics of the multirotor, which is subjected to input and state constraints.

5 Control implementation

Since the objective of the problem formulation was to develop a control strategy for converging the states to a desired value rather than the origin, it is necessary to convert the state model in (17) into error dynamics to accommodate this control strategy. The state errors can be defined as follows: e dep = η dep − η dep d , e r = r − r d , and e in = η in − η in d . Therefore, the error dynamics can be derived as follows:

(38) e ¨ dep e ¨ r e ¨ in = f ¯ dep − η ¨ dep d F ¯ aug − r ¨ d f ¯ in − η ¨ in d + J dep u ¯ dep G ¯ aug u ¯ pos J in u ¯ in + D dep D ¯ aug D in .

For (38), the state constraints X q will have to be converted into equivalent error constraints as follows:

(39) ℰ q = η dep - min − ε dep < e dep < η dep - max − ε dep r min − ε r < e r < r max − ε r η in - min − ε in < e in < η in - max − ε in η ˙ dep - min − ε d -dep < e ˙ dep < η ˙ dep - max − ε d -dep r ˙ min − ε d − r < e ˙ r < r ˙ max − ε d − r η ˙ in - min − ε d − i n < e ˙ in < η ˙ in - max − ε d − i n ,

where ε dep = max ( ∣ η dep d ∣ ) , ε r = max ( ∣ r d ∣ ) , ε in = max ( ∣ η in d ∣ ) , ε d -dep = max ( ∣ η ˙ dep d ∣ ) , ε d − r = max ( ∣ r ˙ d ∣ ) , and ε d − i n = max ( ∣ η ˙ in d ∣ ) . The control algorithm will begin by considering the depth subsystem and stabilize it using the depth information from the LiDAR sensor feedback.

(40) e ¨ dep = f ¯ dep − η ¨ dep d + J dep u ¯ dep + D dep .

By considering the constraints ℰ q and U , the control effort u ¯ dep is obtained using (22) with μ ¯ dep defined as follows:

(41) μ ¯ dep = − J dep − 1 f ¯ dep − η ¨ dep d + λ dep γ ∣ e ˙ dep ∣ sign ( e ˙ dep ) + K dep ∣ S dep ∣ α sign ( S dep ) ] .

The value of K dep ∈ R + can be determined from (35) by heuristically selecting appropriate values for γ , λ dep , and α . The expressions for u ¯ pos and u ¯ in can be derived using a similar approach.

6 Numerical simulations

In this section, the numerical simulations for the proposed approach are presented. The system parameters selected for simulation are detailed in Table 2. It is noteworthy that the value of n is specifically chosen as 1 to showcase the numerical tracking and regulation results.

Table 2

Simulation parameters

Parameters	Value	Parameters	Value
m	0.5 kg	g	10 m/s 2
J x	0.0075 kg m 2	J y	0.0075 kg m 2
J z	0.0150 kg m 2	f	0.1 cm
γ	57/53	α	0.75

Distinct values are assigned to the controller gain λ for each subsystem, and the corresponding value of K is calculated using the relation (35). Taking η dep as the state representing the altitude of the multirotor, the state constraints and input constraints are selected as follows:

(42) X q = 0 < z < 15 − 0.2 1 → 2 < [ u v ] < 0.2 1 → 2 − π 3 1 → 3 < [ ϕ θ ψ ] < π 3 1 → 3 − 1 < z ˙ < 1 − 0.1 1 → 2 < [ u ˙ v ˙ ] < 0.1 1 → 2 − 0.56 1 → 3 < [ ϕ ˙ θ ˙ ψ ˙ ] < 0.56 1 → 3 .

(43) U = − 15 1 → 3 ≤ [ u x u y u z ] ≤ 15 1 → 3 − 1 1 → 3 ≤ [ u ϕ u θ u ψ ] ≤ 1 1 → 3 .

The subsequent subsection presents the results of the regulation and tracking tasks for three cases, specifically:

Case 1: The system is influenced by a conventional terminal sliding mode controller with a power rate reaching law and no constraints.
Case 2: The system is influenced by a conventional terminal sliding mode controller with a power rate reaching law and input saturation only.
Case 3: The system, under the influence of the proposed controller, is subjected to both input and state constraints.

In each scenario, it is assumed that the system is exposed to a bounded lumped disturbance (comprising system uncertainties, external disturbances, and image noises) with magnitudes ∣ D dep ∣ = 0.4 , ∣ D ¯ aug ∣ = 0.03 , and ∣ D in ∣ = 0.2 .

6.1 Regulation task

The goal of the regulation task is to transition the multirotor from the initial pixel coordinates [ u v ] = [ 0.0 0.0 ] T to [ u v ] = [ 0.1 0.1 ] T . Concurrently, the multirotor is required to elevate its altitude from 1 to 2 m and yaw by an angle of 0.5 rad. Figures 5 and 6 illustrate the regulatory performance of the model, with Cases 1, 2, and 3 depicted in blue, red, and black, respectively.

Figure 5

Augmented outer loop states.

Figure 6

Inner loop states.

Analyzing Figure 5 and Table 5, it is evident that the time constants for the responses of u and v are the highest in Case 3, followed by Case 2 and then Case 1. This behaviour is attributed to the introduction of constraints, which tends to slow down the response. However, this trend is not observed in z and ψ . This is because, in the absence of constraints (Case 1), neither z ˙ nor ψ ˙ surpass the desired constraint value. Consequently, when applying system constraints (Cases 2 and 3), the system responses remain unchanged for these states, resulting in unaltered time constants across the three cases. In addition, it is worth noting that the difference in time constants between the three cases is notably small, suggesting that the proposed controller, influenced by X q and U , does not significantly impact the response speed.

Table 5

Time constant comparison

State	Time constant (sec)
	Case 1	Case 2	Case 3
u	0.62	0.64	0.71
v	0.62	0.64	0.71
z	0.91	0.91	0.91
ψ	0.82	0.82	0.82

Tables 3 and 4 detail the maximum values of system states and control effort in both cases. Examination of these tables reveals that the maximum values of system states are considerably higher in Cases 1 and 2 compared to Case 3, especially in states governing the system’s velocity. This observation implies that the addition of input saturation may not effectively constrain the system states to lower values and could potentially contribute to system instability.

Table 3

Comparison table of state variables

Task	Case	Parameters
		z	u	v	ϕ	θ	ψ	z ˙	u ˙	v ˙	ϕ ˙	θ ˙	ψ ˙
Regulation	Case 1	2	0.3	0.2	0.68	0.79	0.5	0.96	0.28	0.28	6.93	7.7	0.53
	Case 2	2	0.3	0.2	0.15	0.21	0.5	0.96	0.28	0.28	4.81	4.18	0.53
	Case 3	2	0.3	0.2	0.12	0.13	0.5	0.96	0.19	0.19	0.5	0.5	0.53
Tracking	Case 1	2	0.3	0.2	1.2	0.2	0.5	0.96	0.22	0.25	12.5	1.32	0.53
	Case 2	2	0.3	0.2	0.27	0.28	0.5	0.96	0.22	0.25	5.69	3.64	0.53
	Case 3	2	0.3	0.2	0.19	0.19	0.5	0.96	0.19	0.19	0.5	0.5	0.49

Table 4

Comparison table of control effort

Task	Case	Parameters
		u x	u y	u z	u ϕ	u θ	u ψ
Regulation	Case 1	35.2	35.2	16	2.82	4.91	0.19
	Case 2	15	15	15	1	1	0.19
	Case 3	15	15	15	1	1	0.19
Tracking	Case 1	24.4	33.9	16	6.85	0.86	0.19
	Case 2	15	15	15	1	0.86	0.19
	Case 3	15	15	15	1	0.86	0.19

A similar pattern is evident in the inner loop states, as depicted in Figure 6. It is noteworthy that the proposed controller restricts the inner loop variables to values significantly lower than the original behaviour, in contrast to Cases 1 and 2. This is particularly significant since a high rate of change in attitude dynamics can induce erratic motions in the system, ultimately leading to instability. In Figure 7, the control effort is presented in the presence and absence of the input constraints U . Without the proposed control law, it is apparent that the control efforts in both the outer and inner loops are notably higher than those under the influence of U .

Figure 7

Control effort.

6.2 Tracking task

In the tracking task, the multirotor is tasked with pursuing a point undergoing circular motion. This is translated into sinusoidal pixel motion characterized by a magnitude of 0.1 and a frequency of 0.15 rad/s. Adjustments to altitude and yaw align with the parameters established in the preceding task. Subsequently, the tracking results for cases 1, 2, and 3 will be graphically illustrated using blue, red, and black representations, respectively.

Figures 8 and 9 illustrate the system states during the tracking task for the three considered cases, while Tables 3 and 4 outline the absolute maximum values of both system states and control efforts, as previously conducted. Similar to the regulation task, as observed in Figures 8 and 9, it is evident that the proposed controller (Case 3) effectively performs the tracking task, closely aligning its response with that of Cases 1 and 2. Despite the previously mentioned response sluggishness, its impact on the tracking behaviour of the system is minimal to negligible.

Figure 8

Augmented outer loop states.

Figure 9

Inner loop states.

The proposed controller’s significance (Case 3) becomes apparent when examining the velocity states of the system. This is evident in the data presented in Table 3 and illustrated in Figures 8 and 9. In Cases 1 and 2, the rate of change in states, particularly the inner loop states, is notably higher than that in Case 3. In addition, the velocity magnitudes exhibit erratic changes in Cases 1 and 2 compared to the more stable behaviour observed in Case 3. This stability is crucial in visual servoing applications, particularly when dealing with dynamic targets. Abrupt increases in system velocities (linear and/or angular) in Cases 1 and 2 may lead to unpredictable movements, potentially causing the system to lose sight of the target being tracked.

As depicted in Figure 10 and corroborated by the data in Table 4, the magnitude of control effort is significantly reduced. This reduction holds potential to alleviate issues related to electronic speed controllers overheating and, concurrently, contribute to an extended flight time. The succeeding section presents the experimental validation of the proposed approach on a real-time hardware for a tracking task.

Figure 10

Control effort.

7 Experimental validation

The proposed approach’s effectiveness is validated by conducting experiments on a quadrotor system, depicted in Figure 11. The quadrotor is built using a DJI F450 frame and includes a carbon fibre landing gear to improve shock absorption during landings. To automate the quadrotor’s flight, the Holybro Pixhawk RPi CM4 Baseboard is utilized as the flight controller. This hardware platform combines the functionalities of the Raspberry Pi Compute Module 4 and the Pixhawk autopilot system. For visual perception, a Logitech BRIO 500 camera is mounted on the quadrotor, facing downwards. This camera has a 4K ultra-high-definition sensor, allowing for the capture of clear and sharp images even in low-light conditions. In addition, to enhance depth estimation, a LiDAR sensor is used in conjunction with the downward-facing camera, operating within the same reference frame.

Figure 11

Hardware setup.

An automatic ground vehicle (AGV) equipped with an ArUco marker is employed to accomplish target tracking. Figure 12 visually depicts the three-dimensional trajectory followed by the quadrotor as it tracks the AGV on the ground. This experiment aims to simulate various scenarios, such as target loss and actuator saturation, by directing the AGV to follow a lemniscate trajectory at different velocities. The proposed NRADC’s effectiveness is assessed and compared to the performance of the nonrecursive controller outlined by Rubagotti and Ferrara [15] alongside the augmented multirotor dynamics within each of these scenarios.

Figure 12

3D trajectory of quadrotor tracking the AGV using NRADC approach.

The controller is configured with γ = 57 ∕ 53 and α = 0.75 , and the task’s input and state constraints are carefully selected as follows:

(44) X q = 0 < z < 15 − 0.2 1 → 2 < [ u v ] < 0.2 1 → 2 − π 3 1 → 3 < [ ϕ θ ψ ] < π 3 1 → 3 − 1 < z ˙ < 1 − 0.1 1 → 2 < [ u ˙ v ˙ ] < 0.1 1 → 2 − 0.56 1 → 3 < [ ϕ ˙ θ ˙ ψ ˙ ] < 0.56 1 → 3 .

(45) U = − 15 1 → 3 ≤ [ u x u y u z ] ≤ 15 1 → 3 − 1 1 → 3 ≤ [ u ϕ u θ u ψ ] ≤ 1 1 → 3 .

In this task, the AGV maintains a constant speed of 1 m/s, while the quadrotor takes off from the AGV and ascends to an altitude of 4 m. Once reaching this altitude, the NRADC algorithm for tracking is activated. The objective of the quadrotor is to continuously track the target along a lemniscate trajectory while adhering to the system’s constraints. Box and whisker plots for the variations in system states and inputs during the tracking process at a speed of 1 m/s are shown in Figures 13 and 14. These plots demonstrate the effectiveness of the proposed NRADC in maintaining both steady-state values (represented by the boxes) and transient values (represented by the whiskers) of the system states and inputs within the specified constraints. Table 6 provides the average control effort (ACE), the average value of control effort between samples ( Δ u ), and the DOC for the control effort u z , comparing the proposed approach with the nonrecursive control presented in by Rubagotti and Ferrara [15] working alongside the augmented multirotor dynamics. The results in the table highlight the reduction in chattering achieved by incorporating the term μ in the proposed control algorithm, resulting in a smoother control profile.

Figure 13

Quadrotor’s augmented dynamical state variables (left) and altitude and inner loop state variables (right).

Figure 14

Quadrotor’s auxiliary outer loop (left) and inner loop (right) control efforts.

Table 6

Performance evaluation for the control effort u z

Index	Formula	Proposed	[15]
ACE	1 T ∫ 0 T ∣ u z ∣ 2 d t	0.6467	2.8261
Δ u z	1 N ∑ k = 1 N ∣ u z k Δ T ‒ u z ( k − 1 ) Δ T ∣ 2	0.3797	14.9131
DOC	Δ u z /ACE	0.5871	5.2770

The results mentioned earlier can be reproduced when tracking speeds are up to 2 m/s. However, if the AGV speeds exceed 2 m/s, there is a risk of losing the target due to the control effort being constrained by U . It is essential to emphasize that this outcome is a consequence of the physical constraints imposed by the quadrotor’s on-board actuators and not a flaw in the control algorithm itself. In the case of using the conventional terminal sliding mode controller with augmented multirotor dynamics, target loss occurs when the AGV speed surpasses 0.75 m/s.

Expanding the scrutiny of the hardware entails a critical consideration of the specified performance evaluation metrics in Table 1. Particularly, these parameters warrant careful examination within the framework of quadrotor tracking of the AGV, spanning speeds from 0.5 to 2 m/s. The associated values for these performance metrics across various speed ranges can be found in Table 7. In addition, a comparative analysis is performed by juxtaposing these values with those obtained through the implementation of the algorithm proposed in [15].

Table 7

Performance metric comparison between the proposed approach and [15] at varying AGV speeds

Index	Proposed				[15]
	0.5	1.0	1.5	2.0	0.5	1.0	1.5	2.0
ISE	0.3895	0.3916	0.3959	0.4126	0.3997	0.4035	0.4298	0.4338
IAE	0.5746	0.5866	0.5909	0.6027	0.7893	0.8519	0.9186	0.9842
ITAE	0.3123	0.3427	0.3702	0.3829	0.9004	0.9326	0.9873	1.2628
ACE	0.5726	0.5884	0.6278	0.6467	1.9741	2.2327	2.6896	2.8261
Δ u z	0.2868	0.3216	0.3591	0.3797	13.4258	13.8997	14.5874	14.9131
DOC	0.5009	0.5466	0.5720	0.5871	6.8010	6.2255	5.4236	5.277

8 Conclusion

This article introduces a NRADC approach tailored for visual servoing applications with multirotors. The methodology establishes a correlation between pixel variations and the augmented dynamics of the multirotor, effectively addressing challenges such as target loss and actuator saturation by framing them as input and state constraint control problems. Complementing the augmented multirotor dynamics, the authors propose a nonrecursive terminal sliding mode-based controller, ensuring strict adherence to system constraints. The nonrecursive nature of the control strategy renders it suitable for resource-constrained platforms and promotes smoother control, thereby extending the lifespan of the multirotor’s actuators.

To assess the efficacy of the proposed approach, the authors conducted extensive numerical simulations encompassing both regulation and tracking tasks. The simulations demonstrate the versatility of the approach and include a quantitative and qualitative comparison. The results indicate that the proposed approach outperforms conventional terminal sliding mode controllers with a power rate reaching law, with and without constraints. Furthermore, real-time experiments were conducted to validate the effectiveness of the approach. These experiments involved successfully tracking a moving AGV at speeds up to 2 m/s, maintaining continuous visual contact with the target, and ensuring that the actuator operations remain within their physical limits. The experimental findings reinforce the robustness and practical applicability of the proposed nonrecursive augmented dynamics control approach for visual servoing with multirotors.

Funding information: This research is supported by A*STAR under its RIE2025 MANUFACTURING, TRADE AND CONNECTIVITY (MTC) INDUSTRY ALIGNMENT FUND -PRE POSITIONING (IAF-PP), with Award No. M23L5a0002.
Author contributions: Both authors have accepted responsibility for the entire content of this manuscript and approved its submission.
Conflict of interest: The authors state no conflict of interest.
Data availability statement: The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

References

[1] Lin J, Wang Y, Miao Z, Zhong H, Fierro R. Low-complexity control for vision-based landing of quadrotor UAV on unknown moving platform. IEEE Trans Ind Inf. 2021;18(8):5348–58. 10.1109/TII.2021.3129486Suche in Google Scholar

[2] Zhang X, Fang Y, Zhang X, Jiang J, Chen X. Dynamic image-based output feedback control for visual servoing of multirotors. IEEE Trans Ind Inf. 2020;16(12):7624–36. 10.1109/TII.2020.2974485Suche in Google Scholar

[3] Huang Y, Zhu M, Zheng Z, Feroskhan M. Fixed-time autonomous shipboard landing control of a helicopter with external disturbances. Aerospace Sci Technol. 2019;84:18–30. 10.1016/j.ast.2018.07.032Suche in Google Scholar

[4] Popova MG, Liu HH. Position-based visual servoing for target tracking by a quadrotor UAV. In AIAA Guidance, Navigation, and Control Conference. 2016. p. 2092. 10.2514/6.2016-2092Suche in Google Scholar

[5] Shi H, Sun G, Wang Y, Hwang KS. Adaptive image-based visual servoing with temporary loss of the visual signal. IEEE Trans Ind Inf. 2018;15(4):1956–65. 10.1109/TII.2018.2865004Suche in Google Scholar

[6] Xu M, Hu A, Wang H. Visual impedance based Human-Robot co-transportation with a tethered aerial vehicle. IEEE Trans Ind Inf. 2023;19(10):10356–65. 10.1109/TII.2023.3240582Suche in Google Scholar

[7] Choi Y, Oh S. Image-goal navigation via keypoint-based reinforcement learning. In: 2021 18th International Conference on Ubiquitous Robots (UR). IEEE; 2021. p. 18–21. 10.1109/UR52253.2021.9494664Suche in Google Scholar

[8] Galeani S, Tarbouriech S, Turner M, Zaccarian L. A tutorial on modern anti-windup design. Eur J Control. 2009;15(3–4):418–40. 10.3166/ejc.15.418-440Suche in Google Scholar

[9] Mulder EF, Kothare MV, Morari M. Multivariable anti-windup controller synthesis using linear matrix inequalities. Automatica. 2001;37(9):1407–16. 10.1016/S0005-1098(01)00075-9Suche in Google Scholar

[10] Singh K, Mehndiratta M, Feroskhan M. Quadplus: design, modelling, and receding-horizon-based control of a hyperdynamic quadrotor. IEEE Trans Aerosp Electron. Syst. 2021;58(3):1766–79. 10.1109/TAES.2021.3133314Suche in Google Scholar

[11] Rubagotti M, Raimondo DM, Ferrara A, Magni L. Robust model predictive control with integral sliding mode in continuous-time sampled-data nonlinear systems. IEEE Trans Autom Control. 2010;56(3):556–70. 10.1109/TAC.2010.2074590Suche in Google Scholar

[12] Xu Q, Li Y. Micro-/nanopositioning using model predictive output integral discrete sliding mode control. IEEE Trans Ind Electron. 2011;59(2):1161–70. 10.1109/TIE.2011.2157287Suche in Google Scholar

[13] Ding S, Park JH, Chen CC. Second-order sliding mode controller design with output constraint. Automatica. 2020;112:108704. 10.1016/j.automatica.2019.108704Suche in Google Scholar

[14] Fu J, Wu QX, Mao ZH. Chattering-free SMC with unidirectional auxiliary surfaces for nonlinear system with state constraints. Int J Innov Comput Inf Control. 2013;9(12):4793–809. Suche in Google Scholar

[15] Rubagotti M, Ferrara A. Second order sliding mode control of a perturbed double integrator with state constraints. In: Proceedings of the 2010 American Control Conference. IEEE; 2010 Jun 30. p. 985–90. 10.1109/ACC.2010.5530711Suche in Google Scholar

[16] Incremona GP, Rubagotti M, Ferrara A. Sliding mode control of constrained nonlinear systems. IEEE Trans Autom Control. 2016;62(6):2965–72. 10.1109/TAC.2016.2605043Suche in Google Scholar

[17] Maayah B, Moussaoui A, Bushnaq S, Abu Arqub O. The multistep Laplace optimized decomposition method for solving fractional-order coronavirus disease model (COVID-19) via the Caputo fractional approach. Demonstr Math. 2022 Dec 31;55(1):963–77. 10.1515/dema-2022-0183Suche in Google Scholar

[18] Maayah B, Arqub OA, Alnabulsi S, Alsulami H. Numerical solutions and geometric attractors of a fractional model of the cancer-immune based on the Atangana-Baleanu-Caputo derivative and the reproducing kernel scheme. Chin J Phys. 2022;80:463–83. 10.1016/j.cjph.2022.10.002Suche in Google Scholar

[19] Kamath AK, Yogi SC, Behera L, Nahavandi S. Vision augmented 3 DoF quadrotor control using a non-singular fast-terminal sliding mode modified super-twisting controller. In: 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE; 2021. p. 2030–6. 10.1109/SMC52423.2021.9659168Suche in Google Scholar

[20] Chen L, Xiao J, Lin RC, Feroskhan M. Angle-constrained formation maneuvering of unmanned aerial vehicles. IEEE Trans Control Syst Technol. 2023;31(4):1733–46. 10.1109/TCST.2023.3240286Suche in Google Scholar

[21] Martinet P, Cervera E. Stacking Jacobians properly in stereo visual servoing. In Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No. 01CH37164). IEEE; May 21 2001. vol. 1. p. 717–22. 10.1109/ROBOT.2001.932635Suche in Google Scholar

[22] Feng Y, Yu X, Man Z. Non-singular terminal sliding mode control of rigid manipulators. Automatica. 2002 Dec 1;38(12):2159–67. 10.1016/S0005-1098(02)00147-4Suche in Google Scholar

[23] Feng Y, Yu X, Han F. On nonsingular terminal sliding-mode control of nonlinear systems. Automatica. 2013 Jun 1;49(6):1715–22. 10.1016/j.automatica.2013.01.051Suche in Google Scholar

[24] Al-Mahasneh AJ, Anavatti SG, Garratt MA. Self-evolving neural control for a class of nonlinear discrete-time dynamic systems with unknown dynamics and unknown disturbances. IEEE Trans Ind Inf. 2019;16(10):6518–29. 10.1109/TII.2019.2958381Suche in Google Scholar

[25] Xu L, Yao B. Adaptive robust precision motion control of linear motors with negligible electrical dynamics: theory and experiments. IEEE/ASME Trans Mechatron. 2001;6(4):444–52. 10.1109/3516.974858Suche in Google Scholar

Received: 2023-09-26

Revised: 2024-01-09

Accepted: 2024-05-29

Published Online: 2024-08-10

This work is licensed under the Creative Commons Attribution 4.0 International License.

Artikel in diesem Heft

https://doi.org/10.1515/nleng-2024-0018

Schlagwörter für diesen Artikel

nonrecursive control; augmented dynamics; visual servoing; multirotor; state constraints; input constraints; target loss; actuator saturation

Creative Commons

BY 4.0