EFFECTIVENESS OF SENTIMENT ANALYSIS FROM OPINIONS USING GENERATIVE AI: A CASE STUDY OF THAI UNDERGRADUATE STUDENTS’ COMMENTS IN ON-DEMAND LEARNING SYSTEMS

Nutthapat  Kaewrattanapat; Jarumon  Nookhong; Martusorn  Khaengkhan; Napasri  Suwanajote; Sittichai  Pintuma

Nutthapat Kaewrattanapat Suan Sunandha Rajabhat University
Jarumon Nookhong Suan Sunandha Rajabhat University
Martusorn Khaengkhan Suan Sunandha Rajabhat University
Napasri Suwanajote Suan Sunandha Rajabhat University
Sittichai Pintuma Suan Sunandha Rajabhat University

Keywords: Sentiment Analysis, Generative AI, Opinions Analysis

Abstract

This research investigates the effectiveness of Generative AI in conducting sentiment analysis on comments from Thai undergraduate students in an on-demand learning system. By designing and applying a novel methodology, the study examines the alignment between human expert sentiment classification and Generative AI predictions. A dataset of 200 comments was analyzed, with sentiments categorized into positive, negative, and neutral classes by both language experts and an AI model, specifically utilizing the capabilities of ChatGPT from OpenAI (gpt-3.5-turbo). The accuracy and efficiency of the AI’s sentiment classification were evaluated using a Confusion Matrix, which revealed an overall accuracy of 73.63%. The results indicated a high level of precision in the positive and negative categories but highlighted discrepancies in the neutral category, underscoring the nuances and challenges inherent in automated sentiment analysis. These findings contribute to the field of AI-driven sentiment analysis by demonstrating both the promise and complexities of utilizing Generative AI in educational settings.