Evolutionary deep belief network with bootstrap sampling for imbalanced class data

Abstract

Imbalanced class data is a frequent problem faced in classification task. Imbalanced class occurs when the classes in the dataset has a huge distribution gap between them. The class with the most instances is called the majority class, while the class with the least instances is called the minority class. it caused the result to be skewed towards the majority class instead. Common techniques to overcome or minimize the negative effect of imbalanced class data are data sampling, algorithm modification or hybrid. Deep learning algorithm is a state-of-the-art part of the machine learning algorithms. It is popularized due to better performances when handling complex and high dimensional data. Deep belief network (DBN) is an example of a deep learning algo- rithm. It is an intricate form of artificial neural network (ANN). It has a deeper layer of neurons that prevents the network from getting stuck when learning from the inputs. It pre-trains the network using Restricted Boltzmann Machine (RBM) and implement backpropagation neural network (BPNN) as a fine-tune step. However, the training time for DBN is longer because of the layers. Also, there is very little apprehension for the exact amount of data or ideal hyperparameters setting to optimize the performance. Due to its complex and deep layers architecture, deep learning needs a lot of training data in order to give good predictions. In this thesis, an optimized DBN is proposed to control the negative outcomes caused by imbalanced class data towards the performance of the algorithm using an evolutionary algorithm. An evolutionary algorithm (EA) is incor- porated to provide the optimum dropout number, learning rate, batch size and iteration number of BPNN for fine-tuning in DBN. Bootstrap sampling is also incorporated in the algorithm structure to minimize the bias of data training samples. These modifications improved the ability to predict more accurate outcomes. To evaluate the performance of Evolutionary DBN with bootstrap sampling, an experimental setup involving imbal- anced class datasets are conducted. The results of Evolutionary DBN with bootstrap sampling performance is collected and documented in the form of performance metrics. The results are then compared to other machine learning algorithms such as DBN, deep neural network (DNN), BPNN and support vector machine (SVM). According to the outcomes, Evolutionary DBN with bootstrap sampling performed better than DBN and other machine learning algorithms in managing the effects of imbalanced class datasets such as accurate predictions and less partiality. The analysis of statistical tests con- ducted at the end of this thesis supports the conclusion of the experiment.