Accuracy Comparison of Support Vector Machine, Random Forest, and K-Nearest Neighbors for Sundanese Speech Classification

Abstract

To support the preservation of the Sundanese language, speech recognition systems based on machine learning canbe developed. This study aims to evaluate and compare the classification performance of Support Vector Machine, Random Forest, and K-Nearest Neighbors which represent margin-based, ensemble-based, and distance-based classification approaches that have been widely applied in speech classification tasks. A secondary dataset consisting of 100 voice recordings was utilized in this research. The study followed the Knowledge Discovery in Database framework, which encompasses data selection, preprocesing, transformation, data mining, and evaluation phases. Feature extraction was performed using the Mel-Frequecy Cepstral Coefficients method. Experimental result demonstrate that the Random Foret algorithm achieved superior performance, reaching 100% accuracy and an Area Under Curve (AUC) of 100%. Meanwhile, K-Nearest Neighbors achieved 87% accuracy with an AUC of 100%, and Support Vector Machine yielded the lowest performance with  67% accuracy and an AUC of 72.89%. Although Random Forest achieved the highest metrics, futher research is required as a perfect 100% score raises concerns regarding model overfitting. To address this issue, utilizing a large dataset is recommended for future studies. Consequently, K-Nearest Neighbors can be considered a more reliable choice in this study, demonstrating robust and stable performance for MFCC based speech classification on smaller dataset.