A Comparative Study of Machine Learning Algorithms for Indonesian Hoax Detection on Social Media

Abstract

Social media platforms have accelerated the spread of misinformation (hoax) in Indonesia, undermining public trust and informed decision-making. This study aims to compare the performance of three text classification algorithms in detecting Indonesian-language hoax content on social media and to identify linguistic patterns distinguishing hoax from valid content. A publicly available corpus of 3,754 Indonesian tweets labeled as hoax or valid was used. Text was preprocessed through case folding, URL removal, and non-alphabetic character removal, then represented using TF-IDF with unigram and bigram features. Three classifiers, Multinomial Naive Bayes, Logistic Regression, and Linear Support Vector Machine, were trained and evaluated on an 80:20 stratified train-test split using accuracy, precision, recall, F1-score, and Area Under the ROC Curve (AUC). Results: Linear SVM achieved the highest accuracy (84.29%) and F1-score (0.9106), Logistic Regression achieved the highest precision (0.9118) and AUC (0.8156), while Naive Bayes achieved the highest recall (0.9967). Feature analysis revealed that debunking-related terms such as hoax, hoaks, mitos, and faktanya were strongly associated with the valid class, since such terms frequently appear in fact-checking tweets rather than in genuinely neutral news content. Conclusion: Model selection for hoax detection should be guided by the operational goal, favoring high-recall models such as Naive Bayes when the priority is minimizing missed hoaxes, and higher-precision models such as Logistic Regression when minimizing false accusations is prioritized; additionally, the reliance on debunking-style vocabulary as a valid-class signal highlights an important limitation of purely lexical models that future research should address.