Evaluation of Convolutional Neural Network and Gradient Boosting Methods for Bug Severity Classification

Recently, deep learning based methods have been proposed to utilize the featureextraction power of deep neural networks to process bug reports. In this paper, we investi-gate Deep Ensemble learning for binary severity classification of bug reports. Due to thesignificance of feature representation for this task, we initially evaluate the effectivenessof word embedding models FastText and Word2Vec. We observe that the FastText modelachieves more generalised results; hence we adopt this model for the Deep Ensemble learningexperiments. We train and evaluate CNN, LightGBM, XGBoost, AdaBoost, and hybridmodels on seven data sets from Eclipse and Mozilla projects. Results analysis shows thatCNN LightGBM consistently outperforms CNN XGBoost by 9.89%, 5.89%, 10.1%, and8.16%, and CNN AdaBoost by 23.92%, 9.23%, 15.31%, and 11.1% in average accuracy,precision, recall, and f-measure, respectively. Similarly, the CNN LightGBM surpasses state-of-the-art approach by 10.69%, 13.6%, 0.16%, and 6.47% in average accuracy, precision,recall, and f-measure, respectively. Overall, the performance results demonstrates that ourchoice of the CNN’s depth with small hyperparameter tuning is a suitable approach. Italso shows that replacing the weak softmax classifier with a more powerful gradient boostclassifier enhances bug severity classification.