Fuzzy-EKF Controller for Intelligent Wheelchair Navigation

1 Introduction

Many disabled people are confronted by problems of handling an electric wheelchair. We cite the problems of obstacle avoidance, reaching a target point, and passage through a door. Several researchers have proposed many navigation aid tools for disabled persons. There is a strong motivation for research in assistance technology to enable people with disabilities to achieve more autonomy. We can then make improvements using intelligent features from robotics, such as the detection and avoidance of obstacles, assistance during passage through doors, and trajectory tracking.

Several studies have been published in this context. In the work of Leishman et al. [12, 22], they developed an assistance to driving a smart wheelchair, “VAHM3.” Their method allows the achievement of autonomous displacements in two modes: “wall following” or “passing through narrow passages.” Furthermore, we find the research of Fredriksson and Hyyppa [4], who equipped the wheelchair “MICA” with a laser scanner, a rate gyro, and a camera. They developed a software system for path following and navigation. Also, Lopes et al. [14] describe a proposed navigation system architecture, system mapping, and method of obstacle avoidance for the control of the electric wheelchair “Robchair.” In addition, several control methods used in mobile robotics [1, 6, 10] are applied in the field of controlling the electric wheelchair. A fuzzy controller was used for determining the robot motion to reach the target, to avoid obstacles, and to navigate.

The location is a major problem in navigation [24]. Hence, several studies were found in this area, such as the work of Demuraa and Komoriya [3] that proposes a self-position method that estimates waypoints using light detection and ranging (LIDAR). We also find many studies using data fusion of visual, odometry, map matching for localization, and GPS [7–9, 11]. In this paper, we are interested on the extended Kalman filter (EKF) for localization and data fusion. Accordingly, we cite the work of Marron et al. [17], who proposed a method based on the EKF that estimates the mobile robot pose by fusing odometric and vision data. Also, Gao et al. [5] employed an EKF in conjunction with a LIDAR for reliable and accurate wheelchair localization. In the work of Lopes et al. [13], the EKF is chosen for the fusion of the odometric data provided by the wheel encoders and the data issued from magnetic markers. The EKF data fusion is used for online pose estimation.

The intelligent wheelchair is a robotic device that is the subject of this research. The design of this wheelchair and the control system ensure navigation without collision, passage through a door, and navigation in a hallway. This paper presents a prototype of an autonomous wheelchair, which aims to increase the mobility of the disabled, specifically those with severe disabilities and visual disturbances.

In our previous works [18, 20, 21], we have started the development of the experimental platform. This platform is an electric wheelchair for the disabled that is equipped with an automatic control system. The control algorithm developed in this paper allows the wheelchair to reach a target point and to avoid obstacles. The EKF is used for improving the localization by fusing the data collected by the encoders and the ultrasonic sensors [19]. The control algorithm is applied on a real electric wheelchair for navigation in indoor environment.

In the following section, we will describe the global system of control.

2 Global Control System for the Electric Wheelchair

The electric wheelchair used in our application is shown in Figure 1. In our previous works [18, 20, 21], we have developed an electronic control system to manage the automatic or semiautomatic mode. The electronic control system is composed of a joystick connected to a power module that is associated with two direct current motors and two batteries of 24 V [16]. The control card is based on a CB405 microcontroller. CB405 allows managing encoders and exchanging data with a computer through an RS232 connection. Also, we developed a card based on a PIC16F877 microcontroller for managing the ultrasonic sensors (SRF04, SRF08) and sending the distances to the computer.

Figure 1:

Standard Electric Wheelchair with Control System.

The global control system shown in Figure 2 is composed of four blocks and a hysteresis switch:

• Electric wheelchair localization using a relative position: we use the kinematic model to calculate the current position of the wheelchair.

• Calculating the theoretical distance between the sensors and obstacles: these distances are necessary for the simulation of obstacle detection.

• Fuzzy controller to reach the target: the principal controller to achieve the objective of displacement.

• Fuzzy controller for obstacle avoidance and EKF for data fusion and localization.

• Hysteresis switch: it is used for switching between the control blocks, and eliminates the generation of many abrupt switching actions in a very short time. Besides, the wheelchair will move away from the obstacle for more security.

Figure 2:

Global Control System.

In the practical test, we use a computer that manages the control algorithm with the MATLAB software. The sensor information is retrieved online using acquisition cards.

In the following, we will describe the kinematic model of the wheelchair; then, we are interested in the development of a fuzzy controller for obstacle avoidance. In the last part, we will apply the data fusion method based on the EKF for improving localization.

3 Kinematic Modeling of the Electric Wheelchair

The kinematic model of the wheelchair is expressed by the following system of equation:

where V_dk is the velocity of the right wheel, V_gk the velocity of the left wheel, V the linear speed, θ the angle of orientation, L the distance between the two driving wheels, and T is the sampling period used (throughout the paper, T = 1 s). This time is necessary to obtain information with sensors and turn the control program.

The distance separating the target and the electric wheelchair is expressed by the following equations:

where

The purpose of the simulation is to test the wheelchair behavior during the navigation. In addition, the simulation can test several cases of navigation and solve problems encountered during navigation in a cluttered environment.

The electric wheelchair is represented by a rectangle that moves in an environment with obstacles. First, we give the coordinates of the starting point, target point, and starting angle. Second, the fuzzy controller will generate the speeds of the right and left wheels. Hence, the wheelchair will be oriented to the target or it will avoid an obstacle. Several cases have been tested for determining the proper parameters of the fuzzy controller. Finally, we validate experimentally the developed controller.

To simulate navigation with obstacle avoidance, it is necessary to calculate the distance between the sensor and obstacles. Hence, we show later the theoretical method used in the simulation program. In the experimental test, we use the distances measured by ultrasonic sensors.

3.1 Calculating Distances between Sensors and Obstacles

The electric wheelchair is equipped with ultrasonic sensors. In the simulation, the distance is calculated using the position of the sensors and the model of the environment. The position of each sensor is x_ci, y_ci, and θ_ci is the orientation, with i being the number of sensors used. The position is given by Figure 3.

Figure 3:

Wheelchair and Sensor Position in the Environment.

The position (X_i, Y_i) and orientation θ_i of the i sensor in the instant k is calculated using the system of equation (5).

(5){Xik = Xk + xcisinθk + ycicosθkYik = Yk − xcicosθk + ycisinθkθik = θk + θci. (5)

The obstacles detected by the sensors are assimilated by a plane, as shown in Figure 4.

Figure 4:

Distances Measured by the Ultrasonic Sensors.

The representation of the plan P^j is given by the three variables Pnj,Pvj and Prj, with j being the number of the plane representing the obstacles faced during navigation. The distance dikj represents the measure of the i ultrasonic sensor, and, using the variables X_ik, Y_ik, θ_ik, and P^j, is given by equation (6):

(6)dikj = Pvj(Prj − XikcosPnj − YiksinPnj). (6)

The distance obtained from equation (6) is the input of the fuzzy controller to avoid an obstacle.

4 Fuzzy Controller for Navigation

In this section, we will develop the fuzzy controller navigation. A fuzzy controller is built using human knowledge. Therefore, the fuzzy sets theory is a good method for managing imprecision and generating a control that will enable the wheelchair to navigate through complex environments [15]. The fuzzy logic control makes possible the implementation of control systems in which decisions are near human behavior, flexible, and non-linear. In this paper, we use the triangular membership function and the method of fuzzy inference of Takagi-Sugeno-Kang (TSK) [23].

This method is used in applications that need the real-time command. This method simplifies the calculation of aggregation for producing a suitable solution more quickly. This fuzzy inference system of TSK introduced a non-fuzzy conclusion rule. We use the fuzzy inference of order zero, and the conclusion is a constant.

4.2 Fuzzy Controller for Obstacle Avoidance

Generally, the environment of wheelchair navigation contains obstacles (wall, table, or chair), as shown in Figure 5.

Figure 5:

Example of Navigation Environment.

The fuzzy obstacle avoidance controller allows the wheelchair to move without collision. The elements of this fuzzy controller are the fuzzy inference tables related to speeds of the left and right wheels, of the membership functions of different measured distances, and fuzzy conclusions. The controller outputs (Figure 6) are defined as follows: Z = 0 (zero), L = 50 (low), M = 150 (medium), H = 200 (high), and VH = 250 mm/s (very high).

Figure 6:

Output Speed Values.

The outputs are linked to the fuzzy inference speeds of the right and left wheels. Table 1 is made from human logic based on the knowledge of the wheelchair movement. In addition, Table 1 includes the fuzzy rules we have synthesized according to the desired behavior of the electric wheelchair.

Table 1:

Fuzzy Inference Table of the Speed of the Right and Left Wheels.

dl

W

W

W

W

W

W

W

W

W

I

I

I

I

I

df

W

W

W

I

I

I

T

T

T

W

W

W

I

I

dr

W

I

T

W

I

T

W

I

T

W

I

T

W

I

Vd

Z

Z

Z

Z

Z

Z

Z

Z

Z

M

M

Z

H

H

Vg

M

H

VH

M

H

VH

M

H

VH

Z

Z

H

Z

L

dl

I

I

I

I

T

T

T

T

T

T

T

T

T

df

I

T

T

T

W

W

W

I

I

I

T

T

T

dr

T

W

I

T

W

I

T

W

I

T

W

I

T

Vd

L

H

VH

M

M

H

VH

M

H

VH

H

H

VH

Vg

VH

Z

M

VH

Z

Z

Z

Z

L

M

Z

M

VH

The membership functions of right distance (d_r), left distance (d_l), and front distance (d_f) described by Figures 7–9, respectively.

Figure 7:

Membership Functions for the Right Distance.

Figure 8:

Membership Functions for the Left Distance.

Figure 9:

Membership Functions for the Front Distance.

The variables [W (weak), I (intermediate), and T (tall)] of membership functions are distances.

For each combination of the input variables, an action is associated with output variables. Examples of fuzzy rules are as follows:

• If (d_l is W) and (d_f is W) and (d_r is I), then (V_d = Z and V_g = H).

• If (d_l is I) and (d_f is I) and (d_r is T), then (V_d = Z and V_g = VH).

Every sampling period, the control algorithm receives the calculated distances. If they are >0.55 m, it is considered that there are no obstacles around the wheelchair. In this case, the hysteresis switch (see Figure 10) selects the fuzzy controller to reach a target. When the distance is <0.45 m, the hysteresis switch passes to the fuzzy controller for obstacle avoidance.

Figure 10:

Hysteresis Switch.

The response time of the system is the time required for the wheels to reach maximum speed. Therefore, at each sampling period, we can determine the variation of each wheel speed. In the experimental section, wheel speeds are measured from the information issued by the encoders. Also, the response time of the system is integrated in the simulation to approximate the real behavior of the wheelchair. In addition, the safety distance is chosen after testing and considered the maximum speed. Switching between the two controllers is managed by the hysteresis function to prevent generating more abrupt switching in a very short time.

4.3 Simulation of the Global Control System

The environment of navigation is related to the reference (O, X, Y). The electric wheelchair always has a position (X, Y) and an angle θ. Several simulations are performed to test the performance of the navigation algorithm with various situational obstacles.

We consider the case when the wheelchair will pass through the door for the person to reach the bed. The starting point is (X= –6 m, Y= –5 m) with an angle of departure π. In this case, the handicapped person can reach the target (X= 2 m, Y= 0 m) with least effort. Figure 11 shows that the desired location is reached with the starting point outside the room. The simulation result shows that the objective is accomplished without blocking, and navigation is done with obstacle avoidance. The distance between the wheelchair and the wall is acceptable for avoiding collision.

Figure 11:

Navigation and Obstacle Avoidance.

We note that, in some cases, the path traveled by the wheelchair is not the shortest. In this case, we can use a method of synchronization between the map and the control algorithm. This solution will serve to avoid a corner situation, in and out movement, and blocking.

In the next section, we improve the quality of control and the precision of displacement by using an EKF.

5 Data Fusion By Using the EKF

In our case, the electric wheelchair for the disabled has two wheels. The displacement is the result of variation of the speeds of the two driving wheels. The system of the movement equations is given by the kinematic model, with V_g being the linear velocity of the left wheel and V_d the linear velocity of the right wheel. The chair position is shown by the state vector X_k = [x_k, y_k, θ_k]^T. The control vector is U_k = [V_gk, V_dk]^T. The velocity measured by the encoders is used by the EKF to estimate the position of the wheelchair. Improved localization is made by the integration of distance measured by ultrasonic sensors and using data fusion by the EKF [19].

6 Simulation of the Global Control System with Localization by the EKF

In this simulation, localization by the EKF allows data fusion issued by different type of sensors. We use the data issued by the encoders and the calculated distances. We remind that the obstacles are a known position in the frame.

Figure 12 shows the result of navigation in the environment with obstacles. The starting position is considered (X= –5 m, Y= –5 m, θ = –π/2) and the target is (X= 8.5 m, Y= 9.5 m). An error on the calculated distances is added to test the localization algorithm by the EKF. In Figure 12, the red curve is plotted by using the odometer data. The blue curve is obtained by data fusion with the EKF by integrating the calculated distances. The error considered in this simulation is 400 mm. In every situation of obstacle detection, the algorithm of localization by the EKF calculates the new position.

Figure 12:

Trajectory Obtained by the EKF in an Environment with Obstacles.

The results obtained by the simulation improve the performance of the global system of navigation. Therefore, the EKF data fusion method can correct location errors derived by the accumulation of measurement errors of the encoders. These results are useful for improving the performance of automatic navigation systems and achieve the goal with minimal error.

We treat now the implementation of the global control system of the electric wheelchair.

7 Experimental Results of the Global Control System

In this section, we will detail the implementation of the obstacle avoidance system to an electric wheelchair for the disabled. The purpose of this system is to provide a more autonomous navigation for disabled people in an indoor environment. The obstacle detection area is composed of three parts. Each part is attached to an ultrasonic sensor to detect the existence of obstacles in the way of the wheelchair. The reliability of the sensors directly affects the uncertainty of the relative localization. The measurement is done in each sampling period to send, in real-time, the distances to the controller. According to measurements, the control system generates the necessary speeds to get around the obstacles. Figure 13 shows the real environment of navigation. The measurement system is composed of two electronic cards for distance acquisition from the ultrasonic sensors and for displacement acquisition from the encoders. The control of the wheelchair is made online by using MATLAB through a serial link. The developed interface displays the wheelchair position during navigation.

• First, we tested the case to reach a target position (X= 1.5 m, Y= 3.6 m). An obstacle is placed in the position (X= 2 m, Y= 2.5 m). The starting point of the electric wheelchair is (X= 1.5 m, Y= 0.5 m, θ = π/2). Figure 14 shows the path traveled by the wheelchair with avoidance of the obstacle existing in its way. When the wheelchair reaches the target position, it stops.

• Second, we will test the fuzzy controller associated with the EKF for localization. The first time, the wheelchair passes through the door. The second time, the control system generates a command for reaching the target (X= 4.4 m, Y= 7 m). The blue curve (see Figure 15) shows the path traveled by the wheelchair from a start position (X= 1.65 m, Y= 0.5 m, θ = π/2). The localization with the EKF is clear when passing through the door and upon detection of the wall by the right sensor. The localization is made during obstacle avoidance. Hence, the localization depends on the availability of distance measurement. Figure 16 shows the environment and navigation steps, such as passing through the door, moving in the hallway, and the result at the end of the course.

Figure 13:

Navigation of Wheelchair with Obstacle Avoidance.

Figure 14:

Experimental Result of Obstacle Avoidance.

Figure 15:

Experimental Results of the Navigation, Obstacle Avoidance, and Localization.

Figure 16:

Picture of the Wheelchair When Navigating in a Real Environment.

8 Conclusion

To ensure the safety of the electric wheelchair user, specific constraints must be taken into consideration to guarantee navigation without obstacle collision. In this context, we used the fuzzy controller for obstacle avoidance simultaneously with an EKF for multisensor data fusion that provides the highest measurement precision and improved performance of the control system.

We have applied the control system on a real electric wheelchair. It is equipped with ultrasonic sensors and two encoders. The ultrasonic sensors were used to detect obstacles regardless of their shape or color. After testing the global control system on this electric wheelchair for the handicapped, we showed the effectiveness of the developed system. In the future, we will take into account trajectory optimization by using a map of the indoor environment. In this case, we can generate multitarget points to decrease cases of obstacle avoidance. Moreover, we plan to develop a cooperative control system, “handicap-wheelchair,” by using voice command or command with eye movement combined with the developed automatic control.