Sentiment Analysis of MyPertamina Customer Reviews on Twitter with Machine Learning and Deep Learning Methods
DOI:
https://doi.org/10.26593/jrsi.v13i1.6958.145-154Keywords:
text mining, machine learning, deep learning, sentiment analysis, SVMAbstract
To ensure that the distribution process of subsidized fuel is more well-targeted, PT Pertamina has developed an application called MyPertamina. The increasing number of MyPertamina users has led to an increasing number of reviews related to the use of MyPertamina. Reviews of MyPertamina fill various social media channels, including Twitter. However, the analysis of user perceptions through social media has not been optimal. Therefore, a better user sentiment mapping is needed. This study was conducted to answer this need by building a text mining model and designing a prototype that can extract and analyze sentiments from tweets related to MyPertamina. This research adopts the CRISP-DM methodology, which consists of business understanding, data understanding, data preparation, modeling, evaluation, and deployment. The data obtained for model development reached 6,920 tweet data. Each data was classified into one of three sentiment categories, namely positive, negative, and neutral. After data preparation, 2,057 data were used for model development. The models tested in this study consist of Support Vector Machine (SVM), Multinomial Naïve Bayes, Gaussian Naïve Bayes, Long Short-Term Memory (LSTM), and Bidirectional Long Short-Term Memory (Bi-LSTM) algorithms. The model that produced the best evaluation score and was selected for prototype development is the SVM model with an accuracy score of 83.74%, weighted precision of 83.96%, weighted recall of 83.74%, and weighted F1-score of 83.72%. The prototype is used for extracting and predicting sentiment for new datasets, which can then be visualized in the form of graphs and word clouds according to the user's needs.
References
Afidah, D. I., Dairoh, Handayani, S. F., Pratiwi, Wijayatun, R., & Sari, S. N. (2022). Sentimen Ulasan Destinasi Wisata Pulau Bali Menggunakan Bidirectional Long Short Term Memory. Jurnal Manajemen, Teknik Informatika, dan Rekayasa Komputer, 21(3), 607-618.
Alaskar, H., & Saba, T. (2021). Machine Learning and Deep Learning: A Comparative Review. Riyadh: Prince Sultan University.
Alghifari, D. R., Edi, M., & Firmansyah, L. (2022). Implementasi Bidirectional LSTM untuk Analisis Sentimen Terhadap Layanan Grab Indonesia. Jurnal Manajemen Informatika (JAMIKA), 89-99.
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaria, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021). Review of Deep Learning: Concepts, CNN Architectures, Challenges, Applications, Future Directions. Journal of Big Data.
Barkved, K. (2022). How To Know if Your Machine Learning Model Has Good Performance, [Online], Diakses dari obviously.ai: https://www.obviously.ai/post/machine-learning-model-performance [2022, 9 March].
Bian, Y., Ye, R., Zhang, J., & Yan, X. (2022). Customer Preference Identification from Hotel Online Reviews: A Neural Network Based Fine-Grained Sentiment Analysis. Computers & Industrial Engineering, 172.
Birjali, M., Kasri, M., & Beni-Hssane, A. (2021). A Comprehensive Survey on Sentiment Analysis: Approaches, Challenges and Trends. Knowledge-Based Systems 226.
Bisio, F., Oneto, L., & Cambria, E. (2017). Sentic Computing for Social Network Analysis. Sentiment Analysis in Social Networks (hal. 71-90). Cambridge: Elsevier.
Feldman, R., & Sanger, J. (2007). The Text Mining Handbook: Advanced Approaches in Analyzing Unstructured Data. New York, NY: Cambridge University Press.
Han, J., Kamber, M., & Pei, J. (2012). Data Mining Concepts and Techniques (3rd ed.). Waltham, MA: Morgan Kaufmann.
Habyba, A. N., Rahmawati, N., & Triwulandari. (2021). Analisis Sentimen Mahasiswa untuk Perbaikan Desain Afektif Ruang Kelas Jurusan Teknik Industri, Universitas Trisakti. Jurnal Rekayasa Sistem Industri, 10(1), 27-34.
Key, B., Kohl, A., Elflein, J., Puri-Mirza, A., Sapun, P., & Cherowbrier, J. Leading countries based on number of Twitter users as of January 2022, [Online], Diakses dari Statista: https://www. statista.com/statistics/242606/number-of-active-twitter-users-in-selected-countries/ [2022, 22 November].
Medhat, W., Hassan, A., & Korashy, H. (2014). Sentiment Analysis Algorithms and Applications: A Survey. Ain Shams Engineering Journal, 5(4), 1093-1113.
Maulana, R., Voutama, A., & Ridwan, T. (2023). Analisis Sentimen Ulasan Aplikasi Mypertamina pada Google Play Store Menggunakan Algoritma NBC. Jurnal Teknologi Terpadu, 9(1), 42-48.
Mustasaruddin, Budianita, E., Fikry, M., & Yanto, F. (2023). Klasifikasi Sentiment Review Aplikasi MyPertamina Menggunakan Word Embedding FastText dan SVM (Support Vector Machine). Jurnal SIstem Komputer dan Informatika (JSON), 526-534.
Nikmah, T., Ammar, M., Allatif, Y., Husna, R., Kurniasari, P., & Bahri, A. (2022). Comparison of LSTM, SVM, and Naive Bayes for Classifying Sexual Harassment Tweets. Journal of Soft Computing Exploration, 3(2), 131 - 137.
Schröer, C., Kruse, F., & Marx Gómez, J. (2021). A Systematic Literature Review on Applying CRISP-DM Process Model. Procedia Computer Science, 181, 526-534.
Sholahudin, A. (2022). Analisis Sentimen terhadap Kebijakan beli Pertalite menggunakan Aplikasi MyPertamina pada Twitter menggunakan Metode Naive Bayes Classifier (NBC). Tangerang: Universitas Muhammadiyah Tangerang.
Xu, G., Meng, Y., Qiu, X., Yu, Z., & Wu, X. (2019). Sentiment Analysis of Comment Texts Based on BiLSTM. IEEE Access 7, 51522-51532.
