Students’ Sentiment Analysis Using Natural Language Toolkit in Machine Learning for Module Evaluation

This paper presents a combination of natural language toolkit (NLTK) in machine learning for sentiment analysis used for module evaluation. The module evaluation is typically done at the end of each module. Dataset of 300 students evaluating each module is conducted with excellent, very good, good, fair, and poor sentiments, delivers valuable perceptions into the overall teaching and lecturing quality and decision making for enlightening methodology of teaching and approaches. This paper demonstrates sentiment analysis model trained using logistic regression algorithm in Machine Learning to evaluate the sentiments given by students in their module evaluation. A study comparison has been done between the proposed model and other sentiment analysis for module evaluation. The results of experiments have been analyzed for decision-making.

Introduction

Nowadays, the teaching system embodies a background that is endlessly improved by an enormous number of low facts generated every second in various formats and most often carried useful and valuable information. Discovering and retrieving the unknown “information” from the ocean of educational data establishes one of the most benefits that sentiment analysis can offer.

Sentiments given by students are a valuable foundation of data not only for evaluating a lecture/module but also for reforming strategies and quality for academic improvement. The sentiment analysis from the students’ evaluation of a module entity was considered.

Sentiment analysis can be applied to a document, sentence, or word level. Though, because of the high number of documents, manual management of sentiments is unfeasible. Hence, digitalized data manipulation is of the great role. Sentiment from the text-based (sentence or document) using natural language processing (NLP). Most of the research published until 2016-2017 used full NLP techniques.

Natural language processing (NLP) technique is used by Sentiment analysis to define the attitude of datasets. Huge amount of data that is produced today is unstructured, which involves processing to produce information.

Some samples of unstructured data are news on social media post, feedback on some topics and search history. The technique of investigating natural language and retrieving meaningful information falls under the field of Natural Language Processing (NLP). The Natural Language Toolkit (NLTK) is used as a commonly used NLP library in Python, to analyze textual data.

In this paper, a dataset of survey done over 300 students to evaluate the module was conducted for NLP with different data cleaning methods. Once the dataset is ready for processing, the training model on pre-classified datasets about module evaluation and use the model to classify the sample datasets into excellent, very good, good, fair and poor sentiments.

Materials and Methods

Sentiment Analysis and Related Work

Sentiment analysis, in general, is an assignment that focuses on discovering and appreciating feelings toward a variable, which can be a topic, event or a person, commonly, the intention of sentiment analysis is to get clear feelings of users, recognize the views they demonstrate, and then locate them into positive, negative, and neutral classes. Natural language and machine learning algorithms are used by sentiment analysis arrangements to bring light on, fetch, and extract figures and opinions from vast amounts of textual information [1].

Mostly, sentiment investigation can be executed in three different classes: the document, sentence, aspect level. Sentiment analysis at the document level intends to bring out the sentiments of users by exploring the document entirely. Sentence level investigation tends to be simple as the target is to identify the polarity of sentences instead of the entire document. Aspect level sentiment investigation emphasize on identifying aspects or attributes expressed in reviews and on classifying the opinions of users towards these aspects.

Sentiment investigation has been commonly useful in different application arena, especially in business and social networks, for different reason. Some well-known sentiment analysis business applications include product and services reviews [2], financial markets [3], customer relationship management [4], and marketing strategies and research, among others. About social media applications, the most known application of sentiment analysis is to control the aspect of a specific brand on Twitter or Facebook [5], and explore the reaction of people given a crisis; e.g., COVID-19 [6]. Another important application domain is in politics [7], where sentiment investigation can be helpful for the campaigns in election of candidates running for political positions.

Nowadays, sentiment analysis has also involved a great deal in research attention in the applied sciences and technology arena. In contrast to the above-mentioned area of business or social networks, which emphasize a single participant, the investigation on sentiment analysis in the education arena considers multiple shareholders of education, including lecturers/tutorials, students, decision-makers, and institutions. Specifically, sentiment investigation is mainly applied to improve lecturing, management, and evaluation by analyzing students’ attitudes and behavior towards courses, platforms, institutions, and teachers.

Referring to past literature, one study [8] on “sentiment analysis (SA) in the education domain focused on detecting the approaches and resources used in SA and identifying the main benefits of using SA on education data”, the study is developed from this paper.

Research Design

Introduction

As depicted on Fig. 1, the following actions were performed in this architecture:

Fig. 1. Design of sentiment analysis process.

• Data gathering: The datasets are gathered from three hundred students from different departments evaluating the module/course aspects, and the data are captured on Google Sheets (which was the problem in making the decision from the data) and loaded into the NLTK package for future use.
• Data cleaning: This process includes removing duplicate data, wrong data, empty cells, and data in the wrong format.
• Training and testing model: In this step, the dataset is trained through a Naïve Bayesian classifier.
• Building testing model: Naïve Bayesian Classifier can be used to build the model.

Performance Measures

There are three commonly used performance measurements: accuracy, sensitivity, and specificity. The accuracy of classifiers is the percentage of correctness of outcome among the test sets exploited in this study. It is defined in (1)–(3). The sensitivity is referred to as the true positive rate, and the specificity is the true negative rate. Both sensitivity and specificity used for measuring the factors that affect the performance are presented in (2) and (3), respectively: (1)Accuracy=TP+TNTP+FP+TN+FN (2)Sensitivity=TPTP+FN (3)Specificity=TNTN+FNwhere True Positive (TP), True Negative (TN), False Positive (FP), False Negative (FN).

Results and Tables

The training and tests were done using Google Colab. The results presented in the Table I are evaluation captured from students evaluating the course at the end of the whole lecture/module. This will help the course owner (lecturer, assistant lecturer, or tutorial assistant) get an overview of the progression of the course.

Table I. Representation of Data Retrieved From the Student with Their Sentiments

S/N	Name	Rating	Review
0	19RP04203	Module leader’s mastery of content	Excellent
1	19RP04692	Quantity and relevance of assessment	Good
2	19RP00165	Quantity and relevance of assessment	Good
3	19RP02249	Linking the content with real life and job situation	Very good
4	19RP09028	Module leader’s mastery of content	Excellent
5	19RP10677	Quantity and relevance of assessment	Good
6	19RP08509	Quantity and relevance of assessment	Good
7	18RP09142	Quantity and relevance of assessment	Good
8	19RP07093	Linking the content with real life and job situation	Very good
9	19RP03090	Linking the content with real life and job situation	Very good

The data that was used in the survey was collected through Google Sheets and received on Google Driver. The following codes have been used to connect to these data:

•

•

Some important libraries have been imported to facilitate the running environment and processing of datasets:

•

•

•

•

•

The machine cannot process the language in its original way accurately; the manipulation of the language in a way that machines can understand is needed, giving a sense to the data through the process called tokenization:

•

Sentiment analysis is a process of identifying an attitude of the phrase, text or word. Training data set to train a model has been generated. Supervised learning machine learning process has been used, which requires associating each dataset with a “sentiment” for training. In this paper, the model used the “excellent”, “very good”, “good”, “fair” and “poor” sentiments:

•

The data are prepared for training the logistic regression classifier class is used:

•

•

The data sets are represented in barh as shown by the Fig. 2.

Fig. 2. Visualization of data for training.

The logistic regression classifier class was used to build the model. Apply the .train() method to train the model and the .accuracy () method to test the model on the testing data. The data tested by the built model are presented in Table II.

Table II. Representation of Tested Data

	Precision	Recall	f1-score	Support
0	0.68	0.81	0.74	9022
1	0.95	0.9	0.93	36861
Accuracy			0.89	45883
Macro avg.	0.81	0.86	0.83	45883
Weighted avg.	0.9	0.89	0.89	45883

Accuracy is defined as the percentage of evaluation from the students in the testing dataset for which the model was correctly able to predict the sentiment. An 88.6% accuracy on the test set is good.

Conclusion

Module evaluation is important in the life of the academic institution, being implemented is vital in developing quality of module delivered. Machine learning algorithms are key factor in nowadays data analysis and visualization for many institutions not only academic and the results from them are helping in decision making and the future development of the company and institutions.

Recommendations

Students Sentiment Analysis using Natural Language Toolkit in Machine Learning for Module Evaluation should be a model and the implementation of this model should be monitored and adjust as necessary.

The study is concentrated on module evaluation using logistic regression; however, a similar study can be carried out in multidimensional area.