Classification of Indonesia False News Detection Using Bertopic and Indobert
Veren Prisscilya1*, Abba Suganda Girsang2
Universitas Bina Nusantara, Indonesia
Email: [email protected]1*, [email protected]2
*Correspondence
|
|
ABSTRACT |
|
Keywords: hoax, deep learning, BERTopic, IndoBERT, text classification. |
In the current global era, the development of technology and information is very rapid, so it is very easy to get information/news from the internet. Because of the ease of getting this information, there is a lot of circulating fake news (hoaxes), the news is not filtered so anyone can spread news that is not clear in content. This can lower a person's credibility in the professional world, cause division, threaten physical and mental health, and can also result in material losses. Based on this, to stop the spread of hoaxes is to detect them as early as possible and block them. This detection can use deep learning methods which are also one of the architectures of transformers, namely a combination of BERTopic which is used to find important words from the news narrative, then the words are combined into the narrative and classified using Indo Bidirectional Encoder Representation from Transformer (IndoBERT). For experiments, the author uses a dataset taken from the kaggle.com website entitled Indonesia False News (HOAX) dataset. This study uses a learning rate of 1e-5, a batch size of 16, and 5 epochs so that the f1-Score results are 92% for validation data and 91% for testing data. |
|
|
|
|
|
|
Introduction
False News has become a serious problem in Indonesia, especially in the social media era. False News can have a significant negative impact on society, from influencing public opinion to triggering social conflict.
Hoaxes are defined as fake news that aims to convince readers to believe the fake news. Every day and even every minute a lot of news appears and spreads on social media, these hoaxes are usually made by groups or individuals who aim for personal interests and can also be caused by certain factors. Hoaxes are being spread in various media in Indonesia both from print media and online media.
The purpose of making hoaxes is to persuade, manipulate, and influence readers to do the opposite or prevent correct actions. Usually using threats, misleading, or making readers believe things that are not real and cannot be confirmed (Rahutomo, Pratiwi, & Ramadhani, 2019).
Hoaxes turned out to be more favored by the public. This is evidenced by the ease with which hoaxes are spread by sharing techniques, especially in social media, such as Instagram LINE, WhatsApp, Facebook, and other media. (Marwan, 2018) According to research by the Ministry of Communication and Information (Kominfo), the cause is because the mainstream media is no longer in favor of the community, and the mainstream media is also less sensitive in absorbing the aspirations of the outside community and tends to be a forum for media owners. So that makes people look for alternative media to satisfy their information needs by looking for sites containing these hoaxes. (Sosiawan & Wibowo, 2020).
Identifying hoaxes is not always easy. Hoaxes are often made to resemble real news, with catchy titles and packaged with convincing narratives. This makes it difficult for people to distinguish which news is true and which news is false.
Therefore, to stop the spread of hoaxes is to detect them as early as possible and block them. The detection is proposed by using deep learning methods with BERTopic and IndoBERT (Indonesia Bidirectional Encoder Representation from Transformers) models. The selection of this method is based on the BERT method which has obtained new state-of-the-art on eleven problems in NLP tasks, one of which is text classification. (Rendragraha, Bijaksana, & Romadhony, 2021). BERTopic is a topic modeling technique based on BERT and TF-IDF in creating clusters that produce easily interpretable topics and important words that describe topics. The BERTopic model uses word context. (Egger & Yu, 2022). For BERTopic document clustering, two UMAP algorithms are used to reduce the dimension of the word embedding results and the HDBSCAN algorithm for document clustering. The document clustering process in the BERTopic model is based on the class-based TF-IDF variant value (c-TF-IDF) in determining the uniqueness of a document compared to other documents. (Grootendorst, 2022) Based on this, BERTopic is expected to support this research to find topics automatically. IndoBERT uses a transformer mechanism, where this mechanism works to learn the relationship between each word in a sentence. IndoBERT uses two mechanisms, namely encoder to read input and a decoder to generate predictions. (Ikram, Sinapoy, Sibaroni, & Prasetyowati, 2023). This research only uses a dataset from Kaggle entitled Indonesia False News (Hoax). The dataset was obtained in the 2015-2020 timeframe so that it represents the situation at that time. An example of news that could be a hoax in the present is "Anis is the Governor of DKI Jakarta", in 2017-2020 it was true news, but in the present, it could be a hoax. The truth or untruth of the news depends on the dataset used, it is necessary to update the dataset to ensure the latest information. The dataset is dynamic and can change over time. Therefore, the results of this study may not fully reflect the current situation.
In their journal (Tandijaya, Liliana, & Sugiarto, 2021) propose BERT classify online news portals, where data obtained from Indonesia news portals, namely from kompas.com and sindonews.com, using web scraping with the help of the BeautifulSoup library, where the data is 6309 data and divided into 80% training data and 20% testing data, then the data will be preprocessed and then classified using the BERT model, wherein the classification, data is converted into an input that can be read by the model, followed by making a data loader to help speed up data retrieval. Next, enter the modeling stage. After the model is created, training and evaluation of the model will be carried out. The pre-trained model will be used to classify new news. The evaluation results obtained with the model built obtained an accuracy of 87.54% by tuning parameters with a learning rate configuration of 5e-05, dropout 0.1, and epoch 4.
Several methods for detecting fake news in Indonesia were proposed by (Isa, Nico, & Permana, 2022) Namely TF-IDF, SVM, Naïve Bayes, and Indobert methods. TF-IDF combined with SVM where TF-IDF is used for feature extraction while SVM is used for classification obtained precision, recall, F1-Score and Accuracy results of 90.00%, TF-IDF combined with Naïve Bayes where TF-IDF is used for feature extraction while naïve bayes for classification precision, F1-Score and accuracy of 83.00% and Recall of 85.00%, for the use of the IndoBERT model obtained precision, recall, F1-score and accuracy results of 94.66%. IndoBERT's weakness is that it requires a lot of data and takes a lot of time to process its data compared to the TF-IDF + SVM and TF-IDF + Naïve Bayes models. IndoBERT takes 15 minutes for training time, which is three times longer than TF-IDF + SVM and fifteen times longer than TF-IDF + Naïve Bayes.
There is also a proposal for the Naive Bayes method to detect fake news in Indonesia by combining it with TF features. (Rahutomo et al., 2019). After carrying out the pre-processing data stage, the dataset will calculate the frequency of word occurrences in the document. After getting the frequency of words in all training documents, calculate the probability value P(ci). Each group has a probability value calculated based on the number of documents in the category per document. Next, sample testing was carried out, followed by static testing, which was carried out on 600 news stories, which resulted in an average accuracy of 82.60%, and finally, dynamic testing, which was carried out by entering the news content into the system. Of the 60 news stories tested, 41 produced the same news classification as the manual mark, and 19 produced a different classification with the manual mark. The percentage of valid results is 68.33%, and hoaxes are 31.67%.
A comparison between the LSTM model and the IndoBERT model was carried out to detect fake news taken from Twitter, followed by a preprocessing process to clean the dataset. (Ikram et al., 2023). After that, the training process is carried out with the LSTM model using Word2Vec, while the IndoBERT model does not need to do the Word2Vec process and directly divides the data using 10 K-Fold Cross Validation. After both models have been processed, the accuracy results of the models are compared. The results show that the IndoBERT model has better performance than the LSTM model. The IndoBERT model achieved an average accuracy of 92.07%, while the LSTM model achieved an average accuracy of 87.54%.
To analyze and predict the validity of news in Indonesian (Nayoga, Adipradana, Suryadi, & Suhartono, 2021) Using a supervised text classification approach. In this journal, several deep learning models are proposed, including LSTM, BI-LSTM, GRU, BI-GRU, and 1D-CNN as well as two conventional classifications, namely SVM and naïve Bayes. Of the five deep learning models and two conventional classifications, the best model for detecting fake news in the Indonesian language is 1D-CNN with 97.90% accuracy.
Research conducted by (Hanifa, Fauzan, Hikal, & Ashfiya, 2021) Conducted a comparison between the LSTM and GRU (RNN) methods for the classification of fake news in Indonesian. The data used for original news is a collection of various online news portals circulating in Indonesia such as detik.com, tribunnews.com, and liputan6.com, while fake news is taken from the turnback hoax.id portal. Thus, a total of 600 news articles were obtained, with a total of 372 original news articles and 228 fake news articles. The data is then classified using the LSTM-RNN model with an accuracy of 38% while using the GRU-RNN model with an accuracy of 38%. Because it gets low accuracy results, then hyperparameter tuning is carried out, obtaining the best accuracy results with epochs: 15, optimizer: rmsprop, and batch size: 64 obtained an accuracy result of 72.50% from the LSTM-RNN model, while GRU-RNN obtained an accuracy result of 64.20%.
Research conducted by (Ramadhan, Adhinata, Segara, & Rakhmadani, 2022) Using Random Forest and logistic regression methods, with the dataset used from the Kaggle dataset using 6,560 news titles, with data division of 70% training data and 30% testing data, using the Random Forest (Entropy) model obtained an accuracy result of 84%, random forest (Gini) obtained an accuracy result of 83%, Logistic Regression obtained an accuracy result of 77%.
The journal made by (Agustina & Hermawati, 2021) Uses a dataset of social media news from the kumparan.com site where the data is taken using a library from Python, namely BeautifulSoap, using the naïve Bayes model, the accuracy result is 81%.
Research conducted by (Haumahu, Permana, & Yaddarabullah, 2021) Uses datasets taken from several online news sites, namely kompas.com, detik.com, cnnindonesia.com, liputan6.com, and turnback hoax. Id which was taken in the 2015-2020 time span of 500 news, which consists of 250 real news and 250 fake news, the data is divided into 80% training data, 20% testing data using the Extreme Gradient Boosting (XGBoost) method obtained an accuracy of 92%.
In the journal (Nurhikam, Syaputra, Rohman, & ..., 2023) Detecting fake election news using the random forest algorithm, using a dataset of 859 news where 670 are factual while 189 are fake news, the data uses data from online news platforms, namely detik.com, liputan6.com, okezone.com, kominfo, turnback hoax. Id. The data is trained using the Random Forest model with balanced training and testing data obtained balanced accuracy results obtained accuracy results of 84.55%.
Based on previous research, it is concluded that 1D-CNN has the highest accuracy result of 97.90% followed by IndoBERT with 94.66%. In this study, the author will use IndoBERT as a model used for the classification of fake news in Indonesian, where IndoBERT can learn complex relationships between words.
Research Methods
Fig. 1 Research Stage
Research Stages
The research stages have been illustrated in Fig. 1 figure it can be seen that several stages will be carried out in this study, starting from dataset collection, and design of the proposed method, and will be evaluated in the final stage to determine whether the method design can be used or not. Each of these stages will be explained in detail in the following subsections.
Dataset
The dataset used in this research is the Indonesia False News (HOAX) Dataset, where this dataset is obtained by downloading from the web www.kaggle.com the dataset is 4,231 training data from the dataset there are 3,465 labeled 1 (fake news), and as many as 766 labeled 0 (real news).
Data Pre-Processing
From the previously obtained dataset, data preprocessing is carried out because the data obtained is not structured and consistent so it is necessary to do some data preprocessing, among others:
Fig. 2 Data Pre-Processing
1) Normalization of non-standard words: this stage is carried out to normalize the data in the previous dataset which has non-standard words converted into standard words so that they are by the Enhanced Spelling (EYD). This is done to make it easier for the system to interpret the words from the dataset so that they do not have ambiguous meanings.
2) Case folding: this stage is carried out because the data obtained is not structured and consistent in the use of uppercase and lowercase.
3) Data cleaning: this stage is carried out to remove several symbols, characters, enter, newline, excess spaces, emojis, and repetition of characters in the sentence.
4) Data splitting: used to divide training data, validation data, and test data. The data division is 80% training data, 10% validation data, and 10% test data.
5) Tokenization: this stage uses the IndoBERT transformers model, where IndoBERT tokenization represents tokens that represent each word. Each token will then be represented
BERTopic
Fig. 3 Topic Categorization
After pre-processing the dataset, we continued with topic categorization using BERTopic (Fig. 3) and searched for important words in the news. BERTopic uses clustering techniques to group topics from words that have similar meanings. Where the clustering technique uses HDBSCAN which is used to group sentence embeddings based on similarity in meaning, each cluster represents the topic obtained.
To get a topic, BERTopic identifies words that represent each topic. After obtaining the TF-IDF calculation results, the words are selected based on their high TF-IDF scores. TF-IDF measures how important a word in a news story is compared to all the news stories in the data set. BERTopic outputs the important words for each topic as well as the TF-IDF score of those words.
The following is an example of topic clustering from 1 topic group using important words that have been calculated with high TF-IDF using BERTopic, as follows:
[(‘indonesia’, 0.4196265), (‘ternyata’, 0.41169137), (‘jakarta’, 0.36498433), (‘kepada’, 0.3534637), (‘kalau’, 0.34819508), (‘beberapa’, 0.3468621, (‘sampai’, 0.3382529), (‘nama’, 0.32713622), (‘orang’ 0.30558452), (‘tentara’, 0.29993442)]
After obtaining the results of topic grouping, the next step is to combine the important words obtained with the narrative.
The following is an example of a sentence that will be grouped to get a topic.
'To the Honorable President of the Republic of Indonesia, I want to ask what is the difference between us and the Rohingya refugees, they give temporary viewing assistance to Indigenous Indonesians, in Bali, one of the provinces that provides the largest income in Indonesia, but with the current incident, it seems that you have forgotten about us, because the Rohingya are the same people as you, so special they are so special, that's all I want to ask'
The keyword/topic of the sentence is: Indonesia, apparently, Jakarta, to, if, some, until, name, person, tantara in English:
'Dear Mr. President of the Republic of Indonesia, I want to ask what is the difference between us and the Rohingya refugees, they give us temporary viewing assistance for Indigenous Indonesians, in Bali, one of the provinces that provides the largest income in Indonesia, but with the current incident it seems that you have forgotten with us because the Rohingya are the same people as you, so special they are just what I want to ask'
Keywords/Topics of the sentence are: Indonesia, apparently, Jakarta, too, if, several, until, name, person, and tantra.
Combination of Topic and Narrative
Next, the merging of topic and narrative is done, for example as follows:
‘<uniq> indonesia ternyata jakarta kepada kalau beberapa sampai nama orang tentara <uniq> kepada yang terhormat bapak presiden ri saya mau tanya apa beda nya kami dengan pengungsi rohingya mereka pak kasih bantuan bertonton sementara untuk warga asli indonesia apa bali salah satu provinsi yang memberikan pemasukan paling besar di indonesia tapi dengan kejadian yang sekarang ini seolah pak lupa dengan kami apa karena rohingya seumat dengan bapak hingga begitu spesial nya mereka Cuma itu saja yang ingin saya tanyakan’
In English :
‘<uniq>Indonesia turns out to be Jakarta to some of the names of soldiers <UNIQ> to the honorable President of the Republic of Indonesia, I want to ask you what is the difference between us and the Rohingya refugees, they give you temporary viewing assistance for indigenous Indonesians, Bali, one of the provinces that provides the largest income in Indonesia, but with the current incident, it seems that you have forgotten with us what because the Rohingya are the same people as you, so they are so special, that's it. All I want to ask'
After combining important words and narratives, then tokenization is carried out using IndoBERT tokenization, tokenization and combining important words with the narrative is carried out on each data that has been divided, namely training data, validation data, and test data. Tokenization using IndoBERT represents tokens that represent each word. Each token will then be represented by a vector.
The following is an example of tokenization from the merged data:
[2, 30521, 300, 678, 1369, 455, 1179, 599, 493, 232, 531, 899, 30521, 636, 4925, 166, 493, 5107, 5875, 469, 119, 1085, 5759, 8133, 43, 9648, 719, 857, 8292, 34, 344, 5875, 5108, 5585, 13841, 57, 1831, 1202, 92, 3], [2, 30521, 300, 678, 1369, 455, 1179, 599, 493, 232, 531, 899, 30521, 5580, 15349, 3320, 7970, 1081, 90, 1874, 1871, 19145, 3],
Fig. 4 Word Combinations in Narration
In fig. 4 describes the merging of important words with the narrative in summary, where the text is clustered using BERTopic so that 12 tokens are obtained, of which the tokens consist of 10 important words and 2 others are special tokens located at the beginning and end, while the narrative is embedded using IndoBERT with a total embedding of 768, after getting the results of clustering with BERTopic and embedding with IndoBERT, then the token is merged, because the merger exceeds the limit of IndoBERT embedding, then the last 12 tokens will be discarded to add the BERTopic clustering results at the beginning of the sentence.
IndoBERT
After tokenizing each data, the model can understand the data. Then called the IndoBERT model. To get good results, several things need to be determined before classifying fake news, namely:
1. Optimizer
2. Learning rate
3. Batch size
4. Epoch
A custom dataset is created to create a data loader and then the data loader is called for training, validation, and testing.
Evaluation
After the data is trained, the performance of the model is evaluated with metrics suitable for classification tasks such as accuracy, precision, recall, and F1-score to see which method can be used for fake news classification with the best results.
Confusion Matrix
After conducting the evaluation, the next step is to obtain the confusion matrix to assess the performance of the classification model on the validation and test data.
Results and Discussion
At this stage, classification testing is carried out using the IndoBERT method where at this stage classification has been tried using a learning rate of 1e-4 with a dropout of 0.5 and obtained an accuracy of 92.10%, while classification without using a learning rate and dropout obtained an accuracy of 87.70%, when a learning rate of 1e-4 is used, an accuracy of 91.30% is obtained. Another experiment with fine-tuning obtained an accuracy of 93.20%.
At this stage, model training is carried out with five scenarios, where the optimizer, learning rate, and batch size used are different, the training data on the data that has been split as much as 3,85 training data, then the data is validated using 423 data validation, the following table shows the results of the scenarios that have been made.
Table 1. Performance Results of Validation
|
|
Scen 1 |
Scen 2 |
Scen 3 |
Scen 4 |
Scen 5 |
|
Optimizer |
AdamW |
AdamW |
RMSprop |
SGD |
SGD |
|
Batch Size |
16 |
32 |
16 |
32 |
32 |
|
Epoch |
5 |
5 |
5 |
5 |
5 |
|
Learning Rate |
0,00001 |
0,001 |
0,00001 |
0,00001 |
0,001 |
|
Accuracy |
85% |
85% |
86% |
82% |
85% |
|
Precision |
87% |
85% |
88% |
88% |
85% |
|
Recall |
97% |
100% |
96% |
91% |
100% |
|
F1-Score |
92% |
92% |
92% |
90% |
92% |
In Table 1 there are performance results on validation data that has been trained, the following validation results have been obtained:
1. In the first scenario the validation data was trained using the AdamW optimizer, Batch Size 16, Epoch 5, Learning Rate 1e5 obtained the results of 85% accuracy, 87% precision, Recall 97%, F1-Score 92%.
2. In the second scenario, the validation data was trained using the AdamW optimizer, Batch Size 32, Epoch 5, Learning Rate 1e3 obtained the results of 85% accuracy, 85% precision, Recall 100%, F1-Score 92%.
3. In the third scenario, the validation data was trained using the RMSprop optimizer, Batch Size 16, Epoch 5, Learning Rate 1e5 obtained the results of 86% accuracy, 88% precision, Recall 96%, F1-Score 92%.
4. In the fourth scenario, the validation data was trained using the SGD optimizer, Batch Size 32, Epoch 5, Learning Rate 1e5 obtained the results of 82% accuracy, 88% precision, Recall 91%, F1-Score 90%.
5. In the fifth scenario, validation data trained using SGD optimizer, Batch Size 32, Epoch 5, Learning Rate 1e5 obtained 85% accuracy, 85% precision, Recall 100%, F1-Score 92%.
After validation with the various parameters above, because the labels on the dataset have very far distribution differences, the parameters that will be used for this research are the first parameters obtained by the fake news accuracy results of 85%, precision of 87%, recall of 97% and F1-Score of 92%, the first parameter was chosen because the precision, recall and F1-Score results are high and can be used for imbalanced data and on imbalanced data it cannot use the accuracy results in this study, so for the results of this study using the results of F1-score and for model testing on testing data will be carried out using the first scenario.
After obtaining the best validation results, then the model is tested again with the selected scenario, namely the first scenario. Where the testing data with a total of 423 is tested again to obtain the final results of the proposed model. After training with 3,385 data (80%) and validation of 423 (10%), the resulting model is tested on testing data with a total of 423 (10%). The following are the evaluation results of validation data and testing data:
Table 2
Validation Evaluation Table
|
Method |
Accuracy |
Precision |
Recall |
F1-Score |
|
BERTopic + IndoBERT |
85% |
87% |
97% |
92% |
|
IndoBERT |
81% |
89% |
87% |
88% |
From Table 2, it can be seen that the results of the BERTopic + IndoBERT model validation evaluation obtained an accuracy of 85%, precision of 87%, Recall of 97%, and F1-Score results of 92%, while the IndoBERT results obtained an accuracy of 81%, precision of 89%, Recall 87% and F1-Score of 88%.
Figure 5 Confusion Matrix Validation BERTopic-IndoBERT
Figure 5 presents the confusion matrix results of the validation data BERTopic-IndoBERT, summarized as follows:
1. True Positive (TP): from the prediction results as many as 11 are original news and the original news value indeed is 11.
2. True Negative (TN): from the prediction results as many as 350 are fake news and it is true, the actual 350 are fake news.
3. False Positive (FP): from the prediction results as many as 9 real news and it turns out that the prediction is wrong, the actual 9 are fake news.
4. False Negative (FN): from the prediction as many as 53 are fake news, it turns out the prediction is wrong, the actual 53 are real news.
Figure 6. Confusion Matrix Validation IndoBERT
Figure 6 presents the confusion matrix results of the validation data IndoBERT, summarized as follows:
1. True Positive (TP): from the prediction results as many as 27 are original news and the original news value indeed is 27.
2. True Negative (TN): from the prediction results as many as 314 are fake news and it is true, actually as many as 314 are fake news
3. False Positive (FP): from the prediction results as many as 45 real news and it turns out that the prediction is wrong, the actual 45 are fake news.
4. False Negative (FN): from the prediction results of 37 fake news, it turns out that the prediction is wrong, the actual 37 are real news.
After validating the model, testing is then carried out on the testing data.
Table 3. Testing Evaluation Table
|
Method |
Accuracy |
Precision |
Recall |
F1-Score |
|
BERTopic + IndoBERT |
85% |
87% |
96% |
91% |
|
IndoBERT |
81% |
89% |
87% |
88% |
From Table 3, it can be seen that the BERTopic + IndoBERT model validation evaluation results obtained an accuracy of 85%, precision of 87%, Recall of 96%, and F1-Score results of 91%, while the IndoBERT results obtained an accuracy of 81%, precision of 89%, Recall 87% and F1-Score of 88%. From these results, it can be concluded that the BERTopic + IndoBERT model gets better results to be used to predict fake news in Indonesia than in IndoBERT.
Figure 7. Confusion Matrix Testing BERTopic-IndoBERT
Figure 7 presents the confusion matrix results of the testing data BERTopic-IndoBERT, summarized as follows:
5. True Positive (TP): from the prediction results as many as 26 are original news and the original news value indeed is 26.
6. True Negative (TN): from the prediction results as many as 334 are fake news and it is true, the actual 334 are fake news.
7. False Positive (FP): from the prediction results as many as 15 are real news and it turns out that the prediction is wrong, the actual 15 are fake news.
8. False Negative (FN): from the prediction as many as 49 are fake news, it turns out the prediction is wrong, the actual 49 are real news.
Figure 8. Confusion Matrix Testing IndoBERT
Figure 8 presents the confusion matrix results of the testing data IndoBERT, summarized as follows:
9. True Positive (TP): from the prediction results as many as 38 are original news and the original news value indeed is 38.
10. True Negative (TN): from the prediction results as many as 305 are fake news and it is true, the actual 305 are fake news.
11. False Positive (FP): from the prediction results as many as 44 real news and it turns out that the prediction is wrong, the actual 44 are fake news.
12. False Negative (FN): from the prediction as many as 37 are fake news, it turns out the prediction is wrong, the actual 37 are real news.
After training with 3.385 data (80%) and validation of 423 (10%), the resulting model was tested on testing data with a total of 423 (10%). The evaluation results with the BERTopic + IndoBERT model get quite good accuracy and precision, but the recall results are quite far from accuracy and precision, this indicates an imbalance in model performance, where the model tends to be better at identifying the majority class than the minority class. However, the F1-score value obtained is quite high, indicating that the model is still able to provide a balance between precision and recall. Thus, despite the imbalance in the imbalanced data, the BERTopic+IndoBERT model still provides good overall performance.
Conclusion
This research shows that clustering important words in news using BERTopic and classifying with IndoBERT can improve the results of fake news classification, especially when the distribution of labels is unbalanced. This study produced a higher F1-Score compared to the IndoBERT model alone, where the F1-Score results obtained in the validation evaluation results were 92% using the BERTopic and IndoBERT models while using the IndoBERT model obtained an F1-Score result of 88% and in the testing evaluation obtained F1-Score on the BERTopic and IndoBERT models obtained a result of 91% while in IndoBERT obtained an F1-Score result of 88%. This shows that BERTopic important word grouping can improve the evaluation results of fake news classification, especially when the label distribution is not balanced.
Bibliography
Agustina, Nova, & Hermawati, Mercy. (2021). Implementasi Algoritma Naïve Bayes Classifier untuk Mendeteksi Berita Palsu pada Sosial Media. Factor Exacta, 14(4), 1979–276. https://doi.org/10.30998/faktorexacta.v14i4.11259
Egger, Roman, & Yu, Joanne. (2022). A Topic Modeling Comparison Between LDA, NMF, Top2Vec, and BERTopic to Demystify Twitter Posts. Frontiers in Sociology, 7(May), 1–16. https://doi.org/10.3389/fsoc.2022.886498
Grootendorst, Maarten. (2022). BERTopic: Neural topic modeling with a class-based TF-IDF procedure. Retrieved from http://arxiv.org/abs/2203.05794
Hanifa, Aini, Fauzan, Sugih Ahmad, Hikal, Muhammad, & Ashfiya, Muhammad Bahrul. (2021). Perbandingan Metode LSTM dan GRU (RNN) untuk Klasifikasi Berita Palsu Berbahasa Indonesia. Dinamika Rekayasa, 17(1), 33. https://doi.org/10.20884/1.dr.2021.17.1.436
Haumahu, J. P., Permana, S. D. H., & Yaddarabullah, Y. (2021). Fake news classification for Indonesian news using Extreme Gradient Boosting (XGBoost). IOP Conference Series: Materials Science and Engineering, 1098(5), 052081. https://doi.org/10.1088/1757-899x/1098/5/052081
Ikram, Muhammad, Sinapoy, Kaer, Sibaroni, Yuliant, & Prasetyowati, Sri Suryani. (2023). JURNAL RESTI Comparison of LSTM and IndoBERT Method. 5(158), 2–6.
Isa, Sani Muhamad, Nico, Gary, & Permana, Mikhael. (2022). Indobert for Indonesian Fake News Detection. ICIC Express Letters, 16(3), 289–297. https://doi.org/10.24507/icicel.16.03.289
Marwan, M. Ravii. (2018). Analisa Penyebaran Berita Hoax Di Indonesia. UG Jurnal, 12(2), 16. Retrieved from http://ravii.staff.gunadarma.ac.id/Publications
Nayoga, Bagas Pradipabista, Adipradana, Ryan, Suryadi, Ryan, & Suhartono, Derwin. (2021). Hoax Analyzer for Indonesian News Using Deep Learning Models. Procedia Computer Science, 179(2020), 704–712. https://doi.org/10.1016/j.procs.2021.01.059
Nurhikam, A. S., Syaputra, R., Rohman, S., & ... (2023). Deteksi Berita Palsu Pada Pemilu 2024 Dengan Menggunakan Algoritma Random Forest. DoubleClick: Journal of Computer and Information Technology, 7(1), 41–50. Retrieved from http://e-journal.unipma.ac.id/index.php/doubleclick/article/view/15456%0Ahttp://e-journal.unipma.ac.id/index.php/doubleclick/article/download/15456/5393
Rahutomo, Faisal, Pratiwi, Inggrid Yanuar Risca, & Ramadhani, Diana Mayangsari. (2019). Eksperimen Naïve Bayes Pada Deteksi Berita Hoax Berbahasa Indonesia. Jurnal Penelitian Komunikasi Dan Opini Publik, 23(1). https://doi.org/10.33299/jpkop.23.1.1805
Ramadhan, Nur Ghaniaviyanto, Adhinata, Faisal Dharma, Segara, Alon Jala Tirta, & Rakhmadani, Diovianto Putra. (2022). Deteksi Berita Palsu Menggunakan Metode Random Forest dan Logistic Regression. JURIKOM (Jurnal Riset Komputer), 9(2), 251. https://doi.org/10.30865/jurikom.v9i2.3979
Rendragraha, Adriansyah Dwi, Bijaksana, Moch. Arif, & Romadhony, Ade. (2021). Pendekatan Metode Transformers untuk Deteksi Bahasa Kasar dalam Komentar Berita Online Indonesia. E-Proceeding of Engineering, 8(2), 3385–3395.
Sosiawan, Edwi Arief, & Wibowo, Rudi. (2020). Kontestasi Berita Hoax Pemilu Presiden Tahun 2019 di Media Daring dan Media Sosial. Jurnal Ilmu Komunikasi, 17(2), 133. https://doi.org/10.31315/jik.v17i2.3695
Tandijaya, Jehezkiel Hardwin, Liliana, & Sugiarto, Indar. (2021). Klasifikasi dalam Pembuatan Portal Berita Online dengan Menggunakan Metode BERT. Jurnal Infra, 9(2), 320–325.