An Effective Optimization-Based Neural Network for Musical Note Recognition

1 Introduction

Music is capable of communicating basic emotions, which are recognized effortlessly in adults regardless of musical training and universally across different cultures [5]. Musical sound separation systems attempt to separate individual musical sources from sound mixtures. The human auditory system gives us the extraordinary capability of identifying instruments being played (pitched and nonpitched) from a piece of music and also hearing the rhythm/melody of the individual instrument being played [6].

There are two common meanings for tuning in music. A theoretical tuning system is a set of pitches that can be used to play music. In modern music, it is usually indicated in beats per minute (BPM); however, before the metronome, words were the only way to describe the tempo of a composition [7]. However, knowledge management in the domain of musical instruments is a complex issue, involving a wide range of instrument characteristics, for instance, the physical aspects of instruments such as different types of sound initiation, resonators, as well as the player-instrument relationship. Representing every type of sound-producing material or instrument is a very challenging task, as musical instruments evolve with time and vary across cultures [6].

Musical pitch estimation is a process to find the musical note pitch or the fundamental frequency (F₀) of a musical signal. Basically, the pitch estimator is used as a part of the transcription system so that the performance of the pitch estimator will affect the overall performance of the transcription system [10] and the onset times of the musical notes simultaneously present in a given musical signal. It is considered to be a difficult problem mainly due to the overlap between the harmonics of different pitches, a phenomenon common in western music, where combinations of sounds that share some partials are referred. Polyphonic musical instrument identification consists of estimating the pitch, the onset time, and the instrument associated with each note in a music recording involving several instruments at a time [14].

The problem of estimating the fundamental frequency or pitch of a periodic waveform has been of interest to the signal processing community for many years. In subspace-based multipitch estimators, however, the cost function is usually multimodal with many local extreme a [12]. Indeed, the ability to perceive, recognize, and perform a musical sequence depends on the preservation of distinct patterns of change in pitch (sometimes referred to as melody) and temporal patterns formed by ratio relationships among interonset intervals (sometimes referred to as rhythm). Rescaling these patterns in absolute terms (within musically reasonable bounds) does not interfere with recognition; rescaling pitch amounts to a change in key, and rescaling rate amounts to a change in tempo [8].

2 Related Work

Vincent et al. [11] have modeled each basis spectrum as a weighted sum of narrowband spectra representing a few adjacent harmonic partials, thus enforcing harmonicity and spectral smoothness while adapting the spectral envelope to each instrument. An NMF-like algorithm to estimate the model parameters and to evaluate it on a database of piano recordings is derived considering several choices for the narrow-band spectra. This algorithm performs similarly to supervise NMF using pretrained piano spectra but improves pitch estimation performance by 6%–10% compared to alternative unsupervised NMF algorithms.

Chanda et al. [2] have designed a system for offline isolated musical symbol recognition. The efficacy of a texture analysis-based feature extraction method was compared to a structural shape descriptor-based feature extraction method coupled with a support vector machine (SVM) classifier. Later, three different kinds of feature selection techniques were also analyzed to gauge the contribution of each feature in the overall classification process.

Sazaki et al. [9] have developed a musical note recognition software using the minimum spanning tree algorithm. This software was developed to help beginners in learning music, especially in recognizing musical notes. The input for this software was musical notes image and the output was information of musical note (i.e. the name of the musical note, the beat’s length, and the sound of the recognized musical note). There were four preprocessing stages involved in this research: sobel edge detection, binarization, segmentation, and scaling. The result from preprocessing was used in training process.

de Jesus Guerrero-Turrubiates et al. [3] have proposed a musical note recognition system based on harmonic modification and artificial neural network (ANN) by taking the audio signals as input from a proprietary database that was constructed using an electric guitar as the audio source. Here, downsampling was applied to convert the signal from 44,100 Hz sampling rate to 2100 Hz and the fast Fourier transform (FFT) was used to obtain the signal spectrum. To enhance the fundamental frequency amplitude, the harmonic product spectrum (HPS) algorithm was implemented. To extract relevant information from the input signal, a dimensionality reduction method based on variances was used. The classification was performed by a feed-forward NN (FFNN) or multilayer perceptron (MLP).

Bhalke et al. [1] have used dynamic time warping (DTW) techniques to recognize Indian musical instruments using 39 MFCC features. Six Indian musical instruments from different families were considered so that a large audio database is collected and recorded in the laboratory at 44.1 kHz sampling frequency. Training and test templates were also generated using 39 MFCC features. Sixty-three different features of instruments were studied including temporal, spectral, and cepstral features. The silence part removal using energy and zero crossing rate and DTW algorithm were also implemented.

3 Problem Statement

Pitch estimation has increased its importance due to the wide variety of applications in different fields (e.g. speech and voice recognition and music transcription, to name a few). A lot of efforts have been made on musical recognition with pitch estimation. The task of pitch estimation is made hard with several numbers of factors. One of the main factors is noise; this is because it can be even “louder” than the original signal. Another problem is to estimate the time of the individual notes, as the pitch amplitude varies with time. On classifying the musical notes, taking into account that real-life audio signals work on structured and unstructured environments, the problem involves factors that seriously affect intelligent systems performance. Issues such as noise, variable amplitude, and tone color were present on the audio files conforming to the data set, although it was observed that the NN was robust enough to discriminate these problems. The blind source separation analysis for a single polyphonic instrument’s audio signals could be performed to obtain the independent sources that can be further classified, but the lack of solution for such drawbacks has motivated the authors to do the research work in this area.

4 Proposed Method

Melodies are characterized as a series of notes, in which each note is an amalgamation of its pitch and a melody is a sequence of pitched sounds with musically eloquent pitch and IOI relationships. In music signals, most frequency apparatuses that initiate in dissimilar musical notes overlap. This ends in an intrinsic ambiguity in the explanation of input signals. In addition, the understanding of musical instruments results in an indispensable problem in the multiple-instrument case. Precisely, if the input music is a real recording rather than a sampler performance, then the empathy based on the conventional approaches, the discriminate investigation, or the template matching becomes demanding because of the feature variability for each note.

The input audio signal is primarily experimented in preprocessing stage and then the noise in the sampled signal is detached by median filter. In the feature extraction stage, two types of features, i.e. residual phase (RP) and mel frequency cepstral coefficient (MFCC), were abstracted from the filtered signal. Last, classification was performed in the extracted features by optimization-based neural network (OBNN) algorithm for musical note recognition.

The overall architecture of musical note recognition systems is revealed in Figure 1.

Figure 1:

Block Diagram of the Proposed Method.

There are three overall processing stages in our projected algorithm: preprocessing, feature processing, and classification.

5 Results and Discussion

All the programs are developed using MATLAB 7.01. The system configuration is Pentium IV processor with 3.2 GHz speed and 1 GB RAM. Different test cases are carried out to show the efficiency of the proposed CSA and hybrid methodology, which are given below.

A database of musical notes of the musician is constructed using various music genres such as blues, electronic, folk country, jazz, pop, and rock. The fundamental frequency of the sample are calculated during preprocessing and then its features are extracted and given as the input to the NN.

To give a comprehensive view of the results, we use four frame-wise evaluation metrics for binary classification: accuracy, recall/sensitivity, and specificity. These metrics can be represented in terms of the number of true positives (TP; the method says it is positive and ground truth agrees), true negatives (TN; the method says it is negative and ground truth agrees), false positives (FP; the method says it is positive and ground truth disagrees), and false negatives (FN; the method says it is negative and ground truth disagrees).

The various performance measures used for the effective identification of musical notes are accuracy, sensitivity, and specificity.

Sensitivity (SN): Statistical measure of performance of the classifier to detect musical notes correctly.

Sensitivity=(Number of correctly detected music)/(Total number of algorithm positive outcomes)

Specificity (SP): Statistical measure of performance of the classifier to detect musical notes wrongly.

Specificity=(Number of wrongly detected music)/(Total number of algorithm negative outcomes)

Accuracy=(Number of correctly detected musical notes)+(Correctly normal states)/(Total number of cases)

In musical note recognition, five different types of music were identified. They are blues, electronic, folk country, jazz, pop, and rock. Table 1 illustrates the accuracy comparison result between the proposed OBNN and existing method effectiveness in detecting music. The existing method’s accuracy values obtained for various music genres are less, whereas the OBNN system achieves high values. Therefore, our proposed method is more effective in identifying music accurately.

The sensitivity evaluation results of different music genres are given in Table 2. The existing method’s sensitivity values obtained for various music genres are also not up to the mark when compared to the proposed method, thus showing effectiveness to detect music correctly.

The specificity evaluation results of different music genres are given in Table 3. The existing method’s specificity values obtained for various music genres are also lesser than the proposed OBNN. Thus, our proposed method is more effective to detect music correctly.

The accuracy comparison of the proposed OBNN and existing NN and Gaussian mixture model-universal background model (GMM-UBM) methods for identifying different music genres is shown in Figure 2. The accuracy of our proposed method is high compared to the existing methods. It represents that OBNN correctly classifies the music.

Figure 2:

Accuracy Comparisons of the Proposed and Existing Methods.

Figure 3 illustrates the sensitivity comparison of the proposed and existing methods. The sensitivity of the proposed method is low compared to the existing method. It indicates that it effectively identifies the music.

Figure 3:

Sensitivity comparisons of the proposed and Existing Methods.

Figure 4 illustrates the specificity comparison of the proposed and existing methods. The specificity of the proposed method is high compared to the existing method. It indicates that it effectively discovers different music signals.

Figure 4:

Specificity Comparisons of the Proposed and Existing Methods.

6 Conclusion

In this study, musical notes were accurately classified. Issues such as noise, variable amplitude, and tone color were present on the audio files conforming to the data set, although it was observed that the NN was robust enough to discriminate these problems. The system was able to recognize the fundamental note of the input chord, demonstrating the validity of the proposed approach and the robustness of the NN to noisy signals.