Economic Impact Due Covid-19 Pandemic: Sentiment Analysis on Twitter Using Naive Bayes Classifier and Support Vector Machine

— Covid-19 is an outbreak caused by severe acute respiratory syndrome. Covid-19 first appeared in Indonesia on March 2, 2020, with two confirmed cases and increased to 1285 cases in 30 provinces. One of the impacts of the Covid-19 pandemic is on the economic aspect, which has experienced a drastic decline in income. This study aims to classify public opinion to determine the level of public sentiment on the economic impact of the Covid-19 pandemic and to identify parameters that influence the accuracy of the sentiment analysis classification model. The methods used in this current research are Lexicon, Support Vector Machine (SVM), and Naive Bayes Classifier (NBC). First, Lexicon is used for scoring and labeling the preprocessed data. Second, SVM is used to classify the sentiment, then find the best accuracy using linear, radial, polynomial, and sigmoid kernels. Third, NBC is used to classify sentiment as a comparison method. The results indicated that 255 tweet data consisted of 44 positive tweets (17.25%), 46 neutral tweets (18.04%), and 165 negative tweets (64.71%). Therefore, it can be inferred that the economic impact on the Indonesian people due to the Covid-19 pandemic has a high negative sentiment value. In the performance, SVM yielded a better accuracy of 100%, precision, recall, and F-measure are 1. This study proves that selecting the kernel type and applying underfitting can improve the accuracy of SVM. Also, SVM can perform well on a small amount of training data.


I. INTRODUCTION
Twitter is a free microblogging social networking service that allows registered users to broadcast short messages called tweets [1].A tweet is one of the main features of Twitter that allows users to write words and opinions that are limited to 280 characters.The writing on the tweet is, of course, followed by the author's positive and negative emotions [2].
The impact of the COVID-19 pandemic is on the economic aspect, which has experienced a drastic decline in income.For example, implementing the large-scale social restrictions (PSBB) policy in Indonesia impacted the production activities from large-scale industries to household-scale businesses [3].Several companies in Indonesia have terminated employment or laid off some of their employees due to decreased income due to the Covid-19 pandemic.Therefore, the poverty rate in several regions in Indonesia continues to increase.For example, Central Sulawesi's poverty rate reached 402,023 people (13.18%).Then it expanded to 14.13% from 402,023 to 458,8450 people due to laid-offs' impact [4].Meanwhile, business actors tend to decrease their income by 82 to 84%, directing to the survey results from 10 to July 26, 2020.8 out of every ten companies generally decreased income [5].
The economic impact for Indonesian due to the Covid-19 pandemic is still a problem because it is still being talked about on various social media, including Twitter.Moreover, Twitter data is free of charge using the Twitter application programming interface (API).Twitter API is a software service that can be connected with other software to provide services to users and developers who will process data.
Sentiment analysis is a computational study of people's opinions and emotions through their entities, events, and attributes.Sentiment analysis is a branch of text mining that extracts information from a media and will be presented more concisely so that it is easy to understand [6].The sentiment analysis research will focus on assertions that sentiment is positive, neutral, or negative [7].Conducting sentiment analysis involves taking into account the opinions stated by online users.Then, the researchers can take these opinions and separate the good, neutral, and negative statements from them.
The objectives of this study are as follows:  To classify emotional text data on Twitter using NBC, SVM, and Lexicon-based. To determine the accuracy of sentiment analysis on text data on the Indonesian economic impact due to the Covid-19 pandemic on Twitter. To find out public opinion on the economic impact of the Indonesian people due to the Covid-19 pandemic by analyzing sentiment on Twitter.

II. MATERIAL AND METHOD
In this section, some existing works are studied first.Then, the dataset used in this study is introduced.Finally, the research method is presented.

A. Related Works
Several attempts have been made to sentiment analysis on Twitter.Romadloni, Santoso, and Budilaksono [8] conducted a study of commuter line transportation using the Naive Bayes Classifier (NBC), K-Nearest Neighbor (KNN), and Decision Tree.The preprocessing covered some stages: converting emoticons, cleansing, case folding, tokenizing, and finally, stemming.This study's results indicated differences in performance that Decision Tree had accuracy, precision, sensitivity, and specificity were 100%.Also, Kurniawan et al. [9] investigated the sentiment analysis of political figures in online news media comments using a comparison of NBC, SVM, and Particle Swarm Classification.The accuracy results using SVM, which is optimized with Particle Swarm, have better accuracy than NBC, which is combined with Particle Swarm, which is 78.4 percent.Meanwhile, Najib et al. [10] conducted an ontology-based sentiment analysis on Twitter for the 2019 Indonesian presidential election campaign using Lexicon-based and SVM.The ontology in this study is the economy with financial, employment, and welfare attributes.The data acquisition process obtained a total of 700,000 tweets, the collection was selected based on economic ontology resulting in 16,998 tweets, and 1600 manual labeling was carried out.The Lexicon-based research results obtained an accuracy of 39%, and the SVM method was 83%.This study shows that SVM has better performance than Lexicon Based.In addition, lexicon-based results show that sentiment on the majority of attributes is neutral.While the SVM results show that sentiment on most attributes (financial and welfare) is positive, and employment is neutral.
Using a lexicon-based methodology and categorizing sentiment into positive, neutral, and negative sentiment classes, Nafan and Amalia [11] conducted a sentiment study on Twitter on Indonesia's economic development.The findings showed 63.6% positive replies, 7.4% negative responses, and 29.0% neutral responses.Because they obtained the most significant findings (63.6%), this study demonstrated that the majority of people had a favorable opinion.
Samah et al. [12] investigated the customer opinion about the service quality of three airlines in Malaysia.Online reviews are taken not only in English but also in Malay.Additionally, they built a web-based dashboard to visualize the best Malaysian airlines.The bilingual analysis of Twitter sentiment used the Naive Bayes algorithm.The test phase showed that tweet sentiment classification achieves 93% accuracy in English and 91% in Malay.Every dashboard feature worked properly and visualized a detailed sentiment analysis.The system usability scale managed to get a score of 94.7%.These results proved that web-based dashboards were 'acceptable' as a good solution for understanding public views of airlines in Malaysia.

B. Dataset
The "tweet" feature on Twitter enables users to post a message or status in text, photos, or video.Written tweets will appear on the user's personal site and are viewable by other users.Twitter has a variety of features that its users can use [13], [14].Data 'tweet' are collected twice, first on August 2, 2021, totaling 1995 data.Second, it was taken for ten days, from September 26 to October 5, 2021, with 1496 data.

C. Research Framework
At this stage, we explain the stages of research that make it more manageable to start until complete research.The research framework is shown in Figure 1.

D. Text Preprocessing
Text preprocessing involves extracting unstructured data into a more comprehensible format using machine learning methods.The text is cleaned up and transformed into a format that computers can process during text preprocessing.The quality of text classification is largely determined by the text preprocessing stage.The researcher now uses the Twitter API to collect and process data using the R programming language [15].In the current study, text preprocessing involves crawling, data conversion, cleaning, normalization, visualization, stemming, tokenizing, and stopword removal.Figure 2 shows the text preprocessing stages.1) Crawling: the process is carried out using R programming on the Rstudio IDE and the rtweet package specifically for retrieving data on Twitter.Crawling uses the keyword 'dampak covid+ekonomi.'There are 90 variables listed in the table header.First, we take six variables consisting of: "user_id," "status_id" (the status ID number generated automatically by Twitter), "created_at," "screen_name" (the name of the Twitter user who wrote the tweet), "text" (the tweet being analyzed), and "source" (the user's device for writing tweets).Table I shows the sample of crawling results which already contain six variables.2) Data Conversion: The process that captured tweet data will be saved in .rdsor .csvformat.Then the lexicon dictionary data in .txtformat can be converted or changed to .rdsformat to facilitate the following text preprocessing process.Figure 3 shows a picture of the data conversion stages.
3) Cleaning: Remove meaningless or unused tweet data, such as usernames, punctuation marks, emojis, numbers, hashtags, URLs, and links.An example of cleaning is shown in Table II.

Tweet Before Cleaning
Tweet After Cleaning @QaillaAsyiqah Awas senjata (covid) makan tuan awas senjata covid makan tuan 4) Normalization is changing non-standard or slang words into standard words, according to Indonesian Dictionary (KBBI).Figure 4 is a flowchart of the normalization stage, and Table III depicts the example of Tweet before and after normalization.

5) Visualization:
The obtained and cleaned Tweet will be displayed in the form of wordcloud.This stage aims to display the number of words often appearing in the tweets data.

6)
Tokenizing: Splitting the words on the tweet and converted into pieces of word tokens.Table IV is an example of a tweet that has been converted into tokenizing form.

E. Lexicon, NBC, and SVM Modeling
Lexicon-based sentiment analysis technique relies on a dictionary of opinion words.The lexicon dictionary aims to determine the sentiment of a sentence, whether the sentence or data contains positive or negative opinions (Table VI).Therefore, the word lexicon containing the opinion and sentiment score is required.In the supervised machine learning approach, some labeled data is required for classifier training [16].
The current study needs three classes in modeling because the model requires an Indonesian lexicon dictionary, which means each word is judged based on positive, negative, and neutral dictionaries.A tweet is considered positive if there are more positive words than negative ones.Conversely, a tweet is judged negative if its negative words outnumber the positive ones.Furthermore, the tweet is considered neutral when both positive and negative words have the same value.As a result, this research model employs three classes.Unsupervised learning, also known as clustering, is classifying unlabeled data into sets, so that related samples belong to the same group and disparate ones do not [17].Naive Bayes Classifier is a machine learning method that uses statistical theory and probability.This method is commonly used in machine learning because it has high accuracy with simple calculations [18].This method is called naive because it assumes that the occurrence of certain features does not depend on the occurrence of other features.This classifier evaluates the categorization status CSV(d, c) as probability | that the document d belongs to the category c and compute by Bayes' theorem.The following is an equation of Bayes' theorem [19]: expresses marginal probability, which does not need to be computed because it is constant for all categories.To compute | requires some assumptions about the document structure d.With the document representation as a feature vector d (w1, w2, …).Because the most common assumption is that all coordinates are independent, and thus SVM is a machine learning algorithm for data classification by finding the optimal hyperplane to separate one class from another based on the distance between classes or decision boundaries.For example, a hyperplane in SVM is able to separate two classes in linear SVM and several classes in non-linear SVM [20].The basic principle of SVM is a linear classifier, where SVM can classify data that can be separated linearly.However, SVM has been developed to overcome non-linear problems by applying the concept of kernel tricks in the workspace [21], [22].The advantages of this method are that it is very fast, effective for text classification problems, and able to classify with a small amount of data [19], [23].The optimal hyperplane can be calculated by the following equation [24]: e.g., data training: , , . . ., , ∈ , ∈ +1, −1 can be separated by hyperplane: Data is divided into two classes, which belongs to class -1 (negative sample) is defined as a vector that satisfies the inequality: Meanwhile, those belonging to class +1 (positive sample) meet the following inequalities: 6) where is the input variable, is the given label, is the value of the field (weight),!is the position of the plane relative to the coordinate center.An optimal hyperplane can separate this set of vectors.When they are separated without error, the distance between the vectors closest to the hyperplane is maximum, as shown in Figure 5. F. Performance Evaluation The confusion matrix is an instrument for visual evaluation in machine learning [25].A confusion matrix can forecast classification outcomes utilizing two dimensions: the form of data extracted from an item and the form of the class that needs to be identified.According to Novaković et al. [26], the confusion matrix comprises rows and columns representing the actual class results or the original class and columns indicating the anticipated class results.Ali et al. [27] stated that a confusion matrix is a table that reports the counts of true positives, false positives, true negatives, and false negatives, which are defined as:  True Positive (TP), the label belongs to the class and is correctly predicted. False Positive (FP), the label is not in the class, although the classifier predicted that it is positive. True Negative (TN), the label is accurately predicted and does not belong to the class. False Negative (FN), the label does belong to the class but is predicted as negative.The confusion matrix with three classes is divided into several metrics: accuracy, error rate, recall, precision, and Fmeasure.
The formulas for Class 1 [27], [28] are as follows: 1) Accuracy: The ratio of positive events whose predictions are correct and negative events whose predictions are correct among the number of verified events.

3) True Positive Rate (Recall):
The ratio of correctly identified positive data.

4) Precision:
The ratio of the correctly predicted positive data to the total data in the positive class.

5) F-measure:
The harmonic mean of recall and precision.

III. RESULTS AND DISCUSSION
This section consists of three parts: visualization, machine learning modeling, and discussion.

A. Visualization
The words often appearing in the data tweets are illustrated in wordcloud, and the frequency can be determined.Figure 6 is the result of word visualization in wordcloud form for economic impact due to Covid-19 pandemic.Table VIII shows words often appearing in the tweets data and sorted by the highest number.

B. Lexicon-Based Modeling
This method requires an Indonesian lexicon dictionary because each word will be assessed based on a positive, negative, and neutral dictionary.Tweets will be considered positive if the tweet has more positive words than negative ones.Conversely, a tweet will be considered negative if there are more negative words in it than positive ones.Moreover, the tweet will be considered neutral if positive and negative words have the same value.Table IX   Next is labeling by determining each tweet's positive, negative, and neutral sentiment classes.If the score is greater than 0, the tweet will be considered positive; if it is less than 0, it will be considered negative; if the score is 0, it will be considered neutral.Table X shows the scoring results that have been labeled sentiment.After executing the R programming language command, Table IX is acquired as a file with the.csv extension.According to the data in this file, there are 165 negative tweets (64.71%), 46 neutral tweets (18.04%), and 44 positive tweets (17.25%).

C. Support Vector Machine Modeling
We classify using the radial, linear, polynomial, and sigmoid SVM kernel parameters.Training and test data are used in the data classification process at this step.Because there are only 255 datasets left after the text preprocessing stage with 1995 data, we employ a ratio of 60:40 for training and testing to prevent overfitting and underfitting in the data.Overfitting occurs when the model is too focused on the training dataset, which results in noise and affects the model's ability to generalize data.Underfitting occurs when the amount of data tested is insufficient [29].After using the lexicon-based method for classification, the results of the Qplot and hyperplane lines in the SVM classification are shown in Fig. 7 and 8.The 255 datasets are split into 153 training and 102 test data with a ratio of 60:40 [28].The probability of dividing the dataset is based on the label on the data so that each data has the same result.Then, the first computation on the SVM kernel (radial, linear, polynomial, and sigmoid) in training data.Next, the second computation is a similar kernel in testing data.Figure 9 shows the feature selection of the SVM kernel (radial, linear, polynomial, and sigmoid).

D. Naive Bayes Classifier Modeling
The results of the confusion matrix calculation and performance using the Naive Bayes Classifier are shown in Tables XVI and XVII.The accuracy of the SVM approach (in particular, the radial, linear, and polynomial kernels) is higher than the NBC method, as shown by Tables XVII, where NBC only achieves 83 percent.

E. Discussion
This research supports and exceeds previous evidence that classifying datasets using radial kernel SVM yields an accuracy of 81.32% [30].Furthermore, Ruslim, Adikara, and Indriati [31] classified data using the Lexicon and SVM could obtain an accuracy of 84%.However, the previous study ratio used a split dataset with 80:20 and 90:10, respectively, which are not considered underfitting and overfitting problems.Therefore, due to underfitting and overfitting problems, the current study focuses on a ratio of 60:40.Based on these two studies, it can be seen that the sentiment analysis classification model using SVM and Lexicon is able to produce higher accuracy of 100% than classification without Lexicon because Lexicon is able to process data labeling automatically or computerized.In general, the Lexicon is capable of producing better part-of-speech (POS) tagging than manual labeling.Although this study uses three classes in the confusion matrix, it still shows the ability to achieve highest performance.

IV. CONCLUSION
Public opinion is unquestionably an essential factor in improving service quality.To achieve optimum results, a variety of machine learning approaches are used.It has been demonstrated that SVM can perform well when classifying text data.The stemming procedure is not applied during the text preprocessing because it could alter the meaning of the words used in this investigation.From 1995 tweets that through text preprocessing become 255, and the applied Lexicon, SVM, and NBC classification, the results of the Indonesian public sentiment on the economic impact of Covid-19 are 17.25% positive, 64.71% negative, and 18.04% neutral.
Future work must look into potential issues with Term Frequency-Inverse Document Frequency (TF-IDF) and other approaches, including K-Nearest Neighbors, to achieve higher accuracy.In addition, the Lexicon can be used with ngram tokenizing.

Fig. 1
Fig. 1 Research Framework Research begins with problem identification by the following research questions:  Are NBC, SVM, and Lexicon-based able to classify Twitter public opinion well? What are the results of public sentiment about the economic impact in a specific time range? How is the performance of NBC, SVM, and Lexiconbased in classifying sentiment Twitter?We looked up information from Twitter and read pertinent literature on NBC, SVM, lexicon, and classification performance during the data collection process.Next, we require a Twitter account to complete the authentication procedure before accessing Twitter data.After creating a Twitter account, we must sign up on the website for Twitter developer access permissions.The token must be authenticated by inputting the obtained API code.The following processes-text processing, classification model, and performance evaluation-are explained in the next subchapter.

TABLE III SAMPLE
OF TWEET NORMALIZATION

TABLE V SAMPLE
OF TWEET STOPWORD REMOVAL Table VII is a classification confusion matrix with three classes.In Table VII, A value represents true positive (TP), while E and I represent true negative (TN).Others are misclassified.As for determining false positives (FP) and false negatives (FN), each class will be different.For example, in Class 1, FP is D + G, while FN is B + C.

TABLE VIII FREQUENTLY
WORDS IN TWEETS is an example of scoring tweet data.

Table
XI is a confusion matrix of SVM radial kernel using 10-fold cross-validation.Similarly, Tables XII to XIV show the confusion matrix values of the linear, polynomial, and sigmoid kernels, respectively.

TABLE XI CONFUSION
MATRIX OF SVM RADIAL KERNEL

TABLE XII CONFUSION
MATRIX OF SVM LINEAR KERNEL

TABLE XIII CONFUSION
MATRIX OF SVM POLYNOMIAL KERNEL

TABLE XIV CONFUSION
MATRIX OF SVM SIGMOID KERNEL

Table
XV shows the result of performance in SVM kernel.From Table XV, it can be highlighted that only the sigmoid kernel has the lowest accuracy, recall, precision, and Fmeasure.