Development of an intelligent controller for sports training system based on FPGA

1 Introduction

Sports competitions are very important in most countries worldwide such as the Olympics and other international tournaments. Training intensity and allocated budgets are the key factors in the advancements of athletes. The great gap among countries’ financial resources plays a major role in helping athletes to win most competitions, thus causing many countries to lose or forfeit participating in such competitions even though those poor countries had a great number of talented athletes. With the ongoing development in the field of technology, most sports are moving towards automated training systems that can be used to enhance the quality of the athletes’ training, especially in terms of providing an automated system that could maintain the training duration and intensity at any time needed [1]. Countries are developing different methods and techniques of sports training in order to win gold, silver, and even bronze medals in various sports and competitions. Currently, many countries are preparing very large budgets to prepare athletes for international competitions. These budgets are resources being spent on developing technologies that reduce the cost and time of training, these advanced technological devices and tools have an enormous impact on the training of athletes in high precision and less time [2]. Intelligent system design and implementation has become a critical aspect in the exploration and design of high-quality products in most technologies [3–5]. The integration of technology with sports had a huge effect on the development of many sports, such as baseball and football. The training methods of athletes in most sports are currently dependent on intelligent systems, so developing an automated system for table tennis is essential, especially in terms of time-saving and athletes body development. The continuous training at the same momentum with different tennis ball speeds will undoubtedly enhance and develop the muscles of the athletes, and automated systems can maintain the stability and intensity of training for a long period, rather than using conventional training methods such as training with other athletes. The proposed system will send the tennis ball at various speeds depending on the hand sign of the athletes; each hand sign was assigned to a different velocity, the athlete will simply show his hand sign to the camera, and the intelligent system will detect the hand gesture and adjust the speed of the tennis balls accordingly. The tennis ball’s various speeds are designed according to a survey conducted by table tennis players. The proposed system will also have a major effect in increasing the development process duration of beginner athletes. However, some challenges still need to be tackled to produce a perfect method for sports training. In this article, an intelligent controller for sports training system based on field programmable gate array (FPGA) has been proposed; many contributions in terms of revolutionizing table tennis have been taken into consideration, such as

1. The system will help table tennis athletes in training individually with different speeds and accuracy.

2. The proposed system can be implemented at a very low cost, and it will have great benefits in time and effort management.

3. Compared to new and traditional training methods, the proposed system will target low-budget tennis players, especially young athletes, to develop their skills at a minimum cost yet efficient way.

The approach of designing the proposed system is typical, but its idea and benefits are novel in that they use artificial intelligence and pulse width modulation to control the intensity of the ball-sending mechanism. The device’s necessity lies in its ease of use and low cost, and its significance stems from the need for the tennis ball-sending machine’s intensity to be easily adjustable.

This study aims to design an intelligent system with low cost so it could be implemented in sports training generally and in tennis specifically. All sports are moving towards integrating smart technology with their training process, so it became essential to develop an intelligent system for table tennis to help tennis athletes with their training without the need for coaches and supervision; the proposed device will reduce the costs of training to the minimum and it is easy to use and install. The tennis athletes will be free to choose the convenient training process, duration, and timing.

To achieve the above aim, the following steps have been taken:

1. To manually collect and prepare datasets for training the neural network, so the system would be able to recognize hand signs;

2. To detect the presence of athletes, sensors will be deployed and calibrated so the device will have the ability to detect the presence of the athletes, and then the training session will start accordingly;

3. To monitor athletes to detect hand signs that would be assigned for each training session intensity, then training the intelligent controller with different speeds of sending tennis balls and assign each speed to a specific sign;

4. To use pulse width modulation to control the motor speed.

This article will discuss thoroughly the most recent related works, later on, the article will demonstrate the material and methods that had been applied to design the proposed system, then the training results and testing methods of each intelligent controller will be demonstrated, and the results and the training progress will be thoroughly discussed to look into the effectiveness and reliability of the proposed system and its conclusion.

2 Related works

There are some previous studies on using artificial intelligence methods in sports. Novatchkov and Baca [6] present a study that concentrated on the application of pattern recognition technologies to the evaluation of practices conducted on training machines. Data were collected utilizing way and cable force sensors connected to a variety of weight machines, allowing for the detection of critical displacement and force determinants throughout training. The modeling achievements so far have shown good performance and prediction results, suggesting the viability and potency of AI techniques in automatically monitoring weight training equipment performances and offering athletes with timely recommendations, but the proposed method did not focus on adjusting the training process for athletes based on their own desire, the AI does not control the training intensity and duration. In a previous study [7], a systematic literature review of smart sports training (SST) is presented. Intelligent data analysis methods that are currently used in the field of SST are presented. Sports areas in which SST is already used are presented, as are processes of training and the maturity of SST methods. Finally, future research directions in the emerging field of SST are proposed; the review article showed the necessity of using AI in sports without going further in the techniques and training process. The human–computer interaction technology of artificial intelligence technology was used in the study of Wang and Park [8] to establish an optimization method and the cognitive model of self-help navigation and hypertext navigation designed for students so that they can acquire the desired theoretical concepts through the cognitive model in the system and accomplish the training task more efficiently. The proposed method was lacking any training process, and it mainly focuses on detecting the suitable athletic training for students according to the health monitor, distance and time. An artificial intelligence associated with big data was used the study of Li and Cui [9]; this article shows how to use a golf-assisted training system to transition from an experience-based sports training method to a human motion analysis method. The swing posture parameters of the trainer and coach were calculated using a human body’s posture prediction, the system was reliable in terms of enhancing the golf player’s swing posture without enhancing the golf player’s ability via intensive training sessions, and the system has only the ability to detect without introducing any further benefits to the athlete. This system’s design goal in the study of Wei et al. [10] is to use artificial intelligence information technology to create a set of network intelligent management systems for college sporting events. Subscribers can log in to the college venue system by installing a browser on their mobile phone or computer; college students can register, the proposed system does not introduce any training methods, but it could be used and integrated with the system proposed in this article to facilitate and manage the time. Connor et al. [11] devised a new method to assist scientists and coaches in planning and controlling the prescriptions for the coaching load of sports persons. Their innovative new method aims to solve the problem of providing the best coaching plans for sportspersons over long periods and how to fit those plans when real-world disruptions stray from the best strategy. The device invented by the authors will be ahead of the random controller and the proportional controller when carried out on the task of describing the training loads and their appropriateness to achieve the goal of the training plan strategy at a higher level; the article did a great work in helping coaches to choose the most suitable training process without any help to the coaches and athletes in terms of suitable training and cost reduction. Li and Li [12] rely on scientific sports coaching, and its benefits: first, with the assistance of sensors, they used the data to analyze the body’s motion task and find the law of motion. The second is based on the neural network form; this article leads to human–computer interplay and controls the motion training operation via deep learning. The third is to realize the emotions of users, better improve performance, route scientific training, and improve the training rank and potential of sportsmen, the article focuses on training monitoring, and no training sessions were introduced. Nie and Jiu [13] proposed a movement recognition system that relied on multi-agent technique movement coaching, and this system had a good reaction on common excerption of linear movement. The system can assist with movement simulation coaching. This article also discussed the application research of multiple agent techniques in smart spread distributor sports coaching simulation systems. Petrivskyi et al. [14] focused on using sensors to reduce power consumption, which will lead to an efficient power consumption mechanism in designed networks. The proposed system in our article also uses sensors to reduce power consumption.

The articles discussed in this section focused on integrating artificial intelligence with sports; no articles were found regarding the integration of tennis table training with artificial intelligence.

The articles above used several techniques to help athletes in their training and development; however, most of the techniques that have been used are expensive and difficult to install. Most tennis training nowadays depend on conventional methods such as training coaches and other players. Even mechanical training devices need some supervision to manage the training session. Developing an intelligent training system at a low cost is not easily obtained especially for low-budget athletes and training centers.

3 Materials and methods

The proposed system is based on supervised artificial intelligence that was trained to detect hand gestures and start the training session accordingly. The AI neural network was trained separately using Matlab, and the final trained network was used in our system. Figure 1 shows the hand gestures that should be used for the training.

Figure 1

Hand signal to determine the speed: a – stop the system; b – very slow system working; c – slow system working; d – medium speed of system working; e – fast speed of system working; f – very fast speed of system working.

The proposed system consists of four main parts, which are listed below and explained in detail. An intelligent controller has been proposed, as shown in Figure 2, for the tennis table training process. It consists of motion sensors, a camera, two intelligent controllers, and motors. These parts will be explained in more detail in the following.

Figure 2

Block diagram of the proposed method.

3.3 Two intelligent controller

The first intelligent controller will process and analyze the captured images to detect hand signs. Then the detected hand signs will be input to the second intelligent controller.

The second controller is integrated into the FPGA, and the second controller is utilized to initiate and close the training sessions according to the signal provided by the motion sensor. At the same time, the speed of the tennis balls will be controlled according to the signal provided by the first controller.

The second intelligent controller will receive two inputs that will be used to create an output of the FPGA. The output varies according to the hand signs values provided by the first controller.

FPGA SPARTAN-3E will assign the intensity of the duty cycle waves.

The duty cycle waves of pulse width modulation (PWM) will control the speed of motors, which will eventually send the ball at various velocities.

The parts of the proposed system are shown in Figure 2. The system could be easily installed and used while reducing costs to a minimum. The design of the proposed system is simple, especially in terms of the hardware components being used. Any table tennis player could operate the system via hand signs, and the automated system will do the rest.

The proposed system was simulated using MATLAB, where each part of the proposed system presented in Figure 2 is trained individually. Then, all parts were gathered into one system to achieve an efficient system that could be used in the training process of the table tennis game.

The proposed system was converted into an HDL code for implementation of FPGA SPARTAN-3E, as shown in Figure 3.

Figure 3

FPGA SPARTAN-3E [15].

An FPGA shown in Figure 3 is used for practically installing the intelligent system because of its reliability, speed, and high efficiency.

The proposed intelligent device is designed and simulated according to the information collected from many table tennis athletes. The device will facilitate the training process of athletes in terms of:

1. The device will not run without any person in front of it.

2. If a person passes by the device, it will activate the device recognition process, but the device will not send any tennis balls.

3. If a hand sign was shown in front of the device camera, it will start sending the balls with variant speeds accordingly.

Two neural network controllers designed to prevent sending the balls accidentally, whether were they passing by a person, an athlete or a normal person.

1. The PWM generated by the FPGA will be inputted into the L293D IC.

2. L293D IC is used to send signals that control the speed of the system motor. There are three types of signals: enable, clockwise, and counterclockwise.

4 Results of training the intelligent controllers

4.1 Hand sign detector and dataset

A real-time dataset was collected via a digital camera using the six hand gestures to define the system signs detection process.

Six hundred images were introduced to the system, 100 persons participated in the data collection process, 6 hand signs were collected from each person so the dataset would be diverse, 450 images were used for the training process, and 150 images were used for testing, as shown in Table 1.

Table 1

Dataset used in training process

Class	No. of samples	Training samples	Augmented for training samples	Test samples
0	100	75	300	25
1	100	75	300	25
2	100	75	300	25
3	100	75	300	25
4	100	75	300	25
5	100	75	300	25

Preprocessing techniques were introduced to the collected images of the dataset, a resizing and filtering process were applied, and the dimension of the images was converted into 256 × 256.

The model is trained with training sets so that the model will be fully trained; later on, the model was tested with a new set of images to evaluate the detection accuracy and reliability of the trained model, and the detection accuracy was 90.05%.

The idea of the proposed system is novel in terms of employing hand detection techniques integrated with artificial intelligence. Devices that send the balls with previously assigned velocities are available in many sports. Yet, none is using the novelist approach proposed in this article.

Figure 4 shows a sample of the hand signs that were introduced to the AI neural network. The hands of different people were used to assure the accuracy of the training.

Figure 4

A sample of the dataset: a, g – dataset for stopping the system; b, h – dataset for very slow system working; c, i – dataset for slow system working; d, j – dataset for medium speed of system working; e, k – dataset for fast speed of system working; f, l – dataset for very fast speed of system working.

The accuracy and efficiency of the artificial intelligent network will increase by increasing the number of images that will be used for the training process, so an augmentation process was used. The augmentation process was based on rotating and zooming the dataset images so that the system would be trained with a vast number of images. The rotation process parameters were from 45 degrees to 135 degrees; hence the system would not fall into the overfitting issue during training.

A convolutional neural network (CNN) schema was created mainly for cataloging hand signs, as shown in Table 2. CNN schema extract features from the hand sign images and changes them to a single trainable structure so that the system would be able to characterize the features efficiently.

Table 2

Convolutional Neural Network Schema

Layers	Input_Volume	Output_Volume	Stride@Kernel
Convolution_1 + ReLU	256 × 256 × 1	256 × 256 × 32	1@3 × 3
Average_Pooling	256 * 256 * 32	64 × 64 × 32	2@2 × 2
Convolution _2 + ReLU	128 × 128 × 1	128 × 128 × 32	1@3 × 3
Average_Pooling	128 × 128 × 32	64 × 64 × 32	2@2 × 2
Convolution_3 + ReLU	64 × 64 × 32	64 × 64 × 64	1@3 × 3
Average_Pooling	64 × 64 × 64	32 × 32 × 64	2@2 × 2
Convolution_4 + ReLU	32 × 32 × 64	32 × 32 × 128	1@3 × 3
Average_Pooling	32 × 32 × 128	16 × 16 × 128	2@2 × 2
Convolution _5 + ReLU	16 × 16 × 128	16 × 16 × 256	1@3 × 3
Max_pooling + Dropout (0.5)	16 × 16 × 256	8 × 8 × 256	2@2 × 2
Fully_connected + ReLU + Dropout (0.5)	16,384	8,192	None
Fully_connected + Soft_Max	8,192	6	None

As mentioned previously, the system is composed of two intelligent controllers. The first controller is designed to determine the speed required for the device to work by relying on a digital camera that takes a picture of the palm, and the speed is determined based on the signal that is made by the hand of the person who wants to make the device, and as shown in Figure 1, which the controller is designed to control five different speeds that turn the device either on or off as needed.

4.2 First controller (Image)

In this controller, when a person is willing to use the system, the camera will take a picture which it determined by the person’s hand. This controller cuts and filters the image to determine the required signal. The system was trained on five commodities, which were limited to the number of fingers of one hand of the person who is using the device. The artificial intelligence feed-forward neural networks algorithm is used in this controller; the proposed network is shown in Figure 5.

Figure 5

Neural network used for image training. (Generated by matlab after the training process of the module).

The proposed network consisted of one input layer, three hidden layers that consist of (27, 12, and 8) neurons in each layer, respectively, and an output layer that contains one neuron. This network is chosen by trial and error, and the activation function type for each layer is determined by trial and error. Figure 6 shows the results of the network training used in the first and approved controller in the system.

Figure 6

Simulation result of image training.

The training will continue intensively until reaching the needed results that are shown in Figure 6. A dataset consisting of several hand sign images was introduced to the system so that the neural network could recognize each hand gesture.

4.3 Second controller

The second controller will take readings from the emotion sensors continuously and process the signal that is analyzed in the first controller to provide duty cycles, which in turn will be the basis for the principles of the PWM, which is the entry into the L293D IC in which the motor controlled in terms of operation. The device and the required speed control (Figure 7) represent the network used in the second controller.

Figure 7

Neural network used for second controller training. (Generated by matlab after the training process of the module).

The network consists of two neurons in an input layer; and three hidden layers that include (10, 5, and 4) neurons, respectively, in each hidden layer, and all information used in the network is approved by trial and error. Moreover, the activation function in each layer is also determined by trial and error. Figure 8 represents the results of the second and adopted controller training in the proposed intelligent system.

Figure 8

Simulation results of second controller training.

The training will continue until reaching the true epochs, which in this case will provide the desired results.

The second controller is designed depending on the passive infrared sensor, which in turn is used to determine the basis for the operation of the device; or turn it off to reduce the energy expenditure that results from the continued operation of the device during the times when the device is not used, and the sensor is used, as shown in Figure 9.

Figure 9

Passive infrared sensor module [16].

The passive infrared sensor is shown in Figure 9; the sensor is responsible for detecting a person and sending electrical signals accordingly.

4.4 Testing

After completing the training process of the intelligent controllers, all the controllers were converted into Simulink blocks in order to examine and facilitate the process of converting them to HDL for downloading them to the FPGA. Figure 10 represents the Simulink blocks of the proposed system.

Figure 10

Simulink block of the proposed system.

Several cases were introduced to the trained controllers to test their efficiency, and the response of the neural networks showed promising results in decision-making to control the speed of the motors.

After completing the connection of all parts with the neural networks that were trained to suit all cases in which the proposed system could work, the results were examined using the MATLAB program as shown in Figure 11, which represents the results for many cases in which the system operates, which in turn provides different speeds as needed, by generating PWM to control the required speed.

Figure 11

Pulse width modulation signals to control system speed: a – motor enable-disable signal; b – motor clockwise rotation signal; c – motor counterclockwise rotation signal.

The signals in Figure 11 represent the duration of the pulse width modulation, which is responsible for providing the motors with the needed electricity based on the intelligent controllers’ decision.

5 Discussion of the neural network training results

A manual dataset of hand signs was introduced to the neural network to train it so it would be able to recognize the hand gestures for each pattern intensity of tennis player training. The system showed promising results in detecting hand signs and starting training sessions accordingly, and the results were obtained after five epochs of training the first controller with a trained network consisting of one input layer, and three hidden layers that consist of (27, 12, and 8) neurons in each layer respectively and an output layer contains one neuron.

Sensors were deployed in this intelligent device so that the system could easily detect the presence of players. As soon as the sensor detects the athlete, it will automatically send electrical voltage to the second controller so the training session will start. This method has great benefits in reducing the consumption of energy and increasing the life cycle of the motors since the system will stay idle until the sensor detects a player's presence. After the second controller starts the training session, the system camera will start detecting the athletes’ gestures. Six hand signs were assigned to the intelligent controller, as shown in Figure 1. As soon as the camera detects a recognizable hand sign, it will capture the image and provide it to the controller.

Sixteen epochs were needed for training the second controller to obtain a train network consisting of two neurons in an input layer; and three hidden layers that include (10, 5, and 4) neurons, respectively, in each hidden layer, and all information that used in the network was approved by trial and error. Moreover, the activation function in each layer was also determined by trial and error.

The proposed system targets low-budget athletes. There are no other authors that targeted such an environment. Also, the hand signs have not been used for controlling such an intelligent system.

One of the major limitations that the proposed system faced is that no other authors focused on such a technique, and also, the system was trained with normal athletes only. For the system to be more authenticated and approved, it should be used by the Olympics professional athletes so that the intensity of the training can be adjusted according to the professional athlete’s expertise.

There are no safe-use features in the system. Accidental hand gestures by any person could trigger the system to start the training session. This disadvantage could be overcome by employing a camera detection system that could be used instead of sensors.

The device is designed to reduce costs and be easy to use in order to contribute to the development of athletes who face difficulties in obtaining huge budgets to qualify them to participate in various international competitions. Adding a face detection system would enhance the system in a way that it would recognize the athlete in front of the system and use previous settings that are memorized for each athlete and start the training process accordingly.

6 Conclusions

The proposed system will majorly impact on revolutionizing the training mechanism of tennis table athletes. The proposed system can be implemented in any tennis table training court, so all types of athletes can easily improve their game and ability. The detection mechanism and training intensity categories are based on the opinion of expert tennis table professional players. The proposed system has been designed and implemented according to the steps listed below:

1. Hand signs were collected manually, and the fingers with palms were extracted from each image to train the controller for detection in various conditions.

2. Passive infrared sensors were used to detect people present, the sensor will automatically change detection signals into electrical voltage according to the presence of the athletes, and the system will start and stop accordingly.

3. The system was simulated via Matlab Simulink, and several cases were provided to test its efficiency of the system. The trained neural network showed accurate promising results in terms of athlete presence detection and hand signs detection. The first controller needed five epochs to get an accurate neural network, and the second controller needed 16 epochs to be accurate.

4. Pulse width modulation is used to control the rotational speed of the motor, which is in charge of the ball's speed. Three signals were used to control the system. The first signal was used to turn on/off the system, the second signal to control the speed of clockwise motor rotation, and the third signal to control the speed of the counterclockwise motor rotation. The results showed the reliability and accuracy of the device in terms of detection and training intensity process.

The proposed system could be enhanced by adding more training intensity. A survey should be conducted of the current athletes that will use the proposed device and all the collected feedback must be summarized to find the essential upgrades and adjustments that would be needed to add to the proposed system features.