A hidden Markov optimization model for processing and recognition of English speech feature signals

2.1 HMM-based English speech recognition

When speech recognition technology is applied, it is generally divided into two stages: the training stage and the recognition stage. There are various types of speech recognition algorithms, but the process of recognizing speech is preprocessing the original speech signal, extracting signal features, training with features (matching calculation), and recognizing results.

The training process of the HMM [12] used in this article is shown in Figure 1.

Figure 1

The training process of HMM.

① The original speech signal is input and preprocessed. The purpose of preprocessing is to convert the original continuous speech analog signal into a discrete speech digital signal [13] and remove some of the interference in the original signal. The speech analog signal is converted into a digital signal using sampling and quantization, and the digital signal is windowed using the Hamming window [14]. The corresponding equation is expressed as follows:

where S _w (n) is the speech digital signal after windowing, s(n) is the original speech digital signal, w(n) is the window function, and N is the length of the digital signal.

② Features are extracted from the preprocessed speech signal. This article adopted the Mel-frequency cepstral coefficient (MFCC) [15] to extract the signal features. The related equations are as follows:

where Y(k) is the frequency domain signal after a fast Fourier transform, y(n) is the time domain signal of the Mel frequency of English speeches, k is the serial number of the sampling point of the time domain signal when converting the time domain signal to the frequency domain signal, up to N, n is the time sampling point of the time domain signal, P(ω) is the instantaneous energy of Y(k), H _m (k) is the frequency response of the triangular filter, m is the serial number of a triangular filter in a set, up to M, f(m) is the center frequency of the mth triangular filter in a set of triangular filters, c(l) is the L-order MFCC feature parameter, and S(m) is the energy spectrum function of the frequency domain signal after the filtering process.

③ The HMM is trained using the MFCC features of the digital signal, and the mathematical model of the HMM is expressed as follows:

where M is the HMM, S is the set of hidden states of the HMM, O is the set of observation vectors, A is the set of jump probabilities of the hidden-state self-hop and next-hop, B is the probability density of the observed values, π is the probability value of the state initialization of the HMM, a _ij is the transition probability of transforming from state i (hidden state s _t−1 at time t − 1) to state j (hidden state s _t at time t), and b _ij (o) is the state probability of the output [16].

④ It is seen from the mathematical model of the HMM, for a given observation sequence O, that the hidden-state S is also fixed, and the values of A, B, and π are unique, i.e., a _ij , b _ij (o), and π determine the HMM. The iterative adjustment is performed on a _ij , b _ij (o), and π to make the output probability P(O|M) of O maximum. The formula is expressed as follows:

where a ˆ i j , b ˆ j k , and π ˆ i are the parameters after one iteration, α t ( i ) and β t ( i ) are the forward and backward probabilities, ξ t ( i , j ) is the probability of being in the state i at time t and being in the state j at time t + 1, and γ t ( i ) is the probability of being in the state i at time t.

⑤ After iteration, whether the HMM satisfies the condition of terminating iteration is determined. If it does, the iteration stops, the parameters of the HMM are output, and the model and the corresponding speech feature data are stored in the speech reference template library; if it does not, it returns to step ④ for iteration.

2.2 The BPNN combined HMM algorithm for recognition

The basic principle of the HMM for speech recognition is based on the statistics of training samples, i.e., the model estimates the hidden state through directly observed sample data; therefore, the established model will not fit the actual states completely and may produce recognition errors in the practical application. In short, the HMM has a strong modeling ability for speech samples but is weak to classify data using modeling, especially when facing a large number of samples [17].

In addition to using the HMM for speech recognition, deep learning neural networks are also used for speech recognition. Deep learning neural networks have the advantages of strong adaptive ability and stronger prediction accuracy with the help of the laws of deep mining. In addition, they can process a large amount of sample data in parallel, i.e., it has more advantageous for big data classification.

The advantage of neural networks in classifying big data can make up for the weakness of HMM in classifying data. First, English speech samples were modeled using an HMM. Then, the English samples were decoded by Viterbi decoding using the established model to obtain the cumulative output probability of the state changes of English speech samples. The cumulative output probability was taken as the input to train a BPNN. The specific process is shown in Figure 2.

Figure 2

Basic flow of the BPNN-improved HMM algorithm for English speech recognition.

① The original English speech signals were postpreprocessed for MFCC feature extraction.

② The parameters in the HMM were iteratively adjusted using the MFCC features of the English speech signals until the termination condition was satisfied.

The aforementioned two steps are HMM modeling for English speech samples. The specific steps and the formulas used are consistent with the training steps of the HMM in Section 2.1, and hence, they are only briefly described here.

③ After model training, the English speech signals were decoded by Viterbi decoding to obtain the cumulative output probability of state changes of the English speech signals.

④ The parameters of the BPNN were initialized, and the cumulative output probability of state changes of the English speech signals were input.

⑤ The cumulative probability was calculated layer by layer in the hidden layer of the BPNN by the following formula:

where x _i is the cumulative output probability of state changes of the input English speech samples, a is the output of every layer, β is the adjustment term of every layer, f(·) is the activation function [18], and ω is the weight between layers.

⑥ The result obtained after layer-by-layer calculation was compared with the actual result of the English speech samples, and the cross-entropy was used as the error between the two results. Whether the error met the requirement, i.e., it converged to stability or iterations reached the set number, was determined. If it did not, the weight parameter in the hidden layer was adjusted reversely, and step ⑤ was repeated; if it did, the training stopped, and the results were output.

The basic process of using the trained BPNN-combined HMM algorithm for English recognition is similar to the training process, but the repeated adjustment of internal parameters is not needed. First, the cumulative output probability is obtained by decoding the English speech samples using Viterbi decoding. Then, the cumulative probability is calculated layer by layer in the trained BPNN to obtain the recognition result.

3.1 Experimental environment

The test configurations were Windows 7 system, I7 processor, and 16 G memory.

3.2 Experimental data

English speech data were crawled from the Internet [19], and 10,000 speeches with clean and standard pronunciations were selected, of which 90% were used as the training set and 10% as the test set. The sampling rate was set as 16 kHz, and 16-bit encoding was used. Some sentences were as follows:

① It’s a nice day today;

② How can I get to the airport, please;

③ Wha’s the price of this product

…

3.3 Experimental setup

The design of structural parameters for the HMM is as follows: the number of states of the speech samples was set as 6 by orthogonal comparison experiments, and the number of possible observations for every state was 4. The initial state distribution transition matrix is given as follows:

The relevant parameters of the BPNN are set as follows according to the data volume of single detection and orthogonal experiments. The number of nodes in the input layer was set as 1,024, the number of nodes in the output layer was set as 512, the number of nodes in the hidden layer was set as 2048, the learning rate was set as 0.2, and the error threshold was set as 10⁻⁷.

To verify the performance of the BPNN-improved HMM algorithm, it was compared with the HMM algorithm and the BPNN algorithm.

The recognition performance of the three speech recognition algorithms was compared under the different number of test samples. In addition, Gaussian white noise was added to the test set containing 1000 samples to compare the performance of the speech recognition algorithms for test samples with different signal-to-noise ratios and to test the anti-interference abilities of the three recognition algorithms.

3.4 Evaluation indicators

The results obtained after recognition by the speech recognition algorithms were evaluated by the word error rate [20], and the calculation formula is:

where X is the number of substituted words, Y is the number of deleted words, Z is the number of inserted words, and P is the total number of words.

3.5 Experimental results

Figure 3 shows the word error rates of the three speech recognition algorithms under different numbers of test samples. Figure 3 shows that the word error rate of all three speech recognition algorithms tended to decrease and eventually tended to be gentle as the number of test samples increased.

Figure 3

Word error rates of three speech recognition algorithms under different number of test samples.

Figure 3 shows that the word error rate of the BPNN-improved HMM algorithm for English speech recognition was always lower than that of the other two recognition algorithms; the word error rate of the BPNN algorithm was higher than that of the HMM when the number of test samples was less than 400, and the word error rate of the HMM algorithm was higher than the BPNN algorithm when the number of test samples was more than 400.

Figure 4 shows the time consumption of the three speech recognition algorithms for speech recognition under the different number of test samples. Figure 4 shows that as the number of test samples increased, the time required by all three recognition algorithms increased. When the number of test samples was less than 400, the recognition time of the three algorithms was relatively close; when it exceeded 400, the gap between the recognition time of the three algorithms became larger, and the increased amplitude also increased. Overall, the HMM algorithm had the longest recognition time, the BPNN had the second-longest recognition time, and the BPNN-improved HMM algorithm had the lowest recognition time under the same number of test samples.

Figure 4

Recognition time consumption of three speech recognition algorithms under different number of test samples.

Figure 5 shows that the word error rate of all three recognition algorithms tended to decrease as the signal-to-noise ratio of the speech signal increased, which meant that the smaller the noise interference was, the lower the word error rate was. Under the same signal-to-noise ratio, the HMM algorithm had the highest word error rate for speech recognition, followed by the BPNN algorithm and the BPNN-improved HMM algorithm.

Figure 5

Word error rates of three speech recognition algorithms under different signal-to-noise ratios.

Figure 6 shows that the recognition time of the three speech recognition algorithms reduced as the signal-to-noise ratio increased. The reason for the aforementioned result is that when the signal-to-noise ratio was higher, there was less interference in the speech signal, and the algorithm was less hindered in recognition. Under the same signal-to-noise ratio, the HMM algorithm had the longest recognition time, followed by the BPNN algorithm and the BPNN-improved HMM algorithm.

Figure 6

Recognition time consumption of three speech recognition algorithms under different signal-to-noise ratios.