Ontology-Driven Sentiment Analysis of Product and Service Aspects
Author(s): Kim Schouten, Flavius Frasincar
Full text: submitted version
Abstract: With so much opinionated, but unstructured, data available on the Web, sentiment analysis has become popular with both companies and researchers. Aspect-based sentiment analysis goes one step further by relating the expressed sentiment in a text to the topic, or aspect, the sentiment is expressed on. This enables a detailed analysis of the sentiment expressed in, for example, reviews of products or services. In this paper we propose a knowledge-driven approach to aspect sentiment analysis that complements traditional machine learning methods. By utilizing common domain knowledge, as encoded in an ontology, we improve the sentiment analysis of a given aspect. The domain knowledge is used to determine which words are expressing sentiment on the given aspect as well as to disambiguate sentiment carrying words or phrases. The proposed method has a highly competitive performance of over 80% accuracy on both SemEval-2015 and SemEval-2016 data, significantly outperforming the considered baselines.
Keywords: aspect-based sentiment analysis; product and service reviews; domain ontology; SVM
Review 1 (by anonymous reviewer)
(RELEVANCE TO ESWC) The paper integrates a semantic strategy. This makes it sound for ESWC. (NOVELTY OF THE PROPOSED SOLUTION) The novelty of the proposed approach is limited but interesting with respect to the state of the art. (CORRECTNESS AND COMPLETENESS OF THE PROPOSED SOLUTION) The presented approach is correct and complete with respect to the problem they want to address. (EVALUATION OF THE STATE-OF-THE-ART) The related work should be slightly improved as mentioned in the general review. (DEMONSTRATION AND DISCUSSION OF THE PROPERTIES OF THE PROPOSED APPROACH) Discussion is proper. (REPRODUCIBILITY AND GENERALITY OF THE EXPERIMENTAL STUDY) Results are reproducible. (OVERALL SCORE) The paper discusses an approach mixing semantic and statistical strategies for improving the effectiveness of aspect-based sentiment analysis systems. The idea behind this paper is timely since the way reviews are written by users needs the capability of extracting from text the set of pairs <aspect, opinion-words> in order to analyze them separately. Actually, I don't have any major concern about this paper expect for some minor stuff. 1) Ontology analysis. In order to better provide a final judgement about the contribution of this paper, it would be helpful to give access to the ontology in order to perform a more detailed analysis. From the paper the ontology seems quite simple, but even this is not a problem for the Reviewer, it would be interesting to analyze it in order to provide (eventually) some feedbacks that might be of help for improving it. 2) Related work. The related work shoud be better linked with aspect-based contribution proposed with, for example, the recent editions of the workshop on sentiment analysis run at ESWC in the last years. A brief recap should be provided. 3) Evaluation. By considering the particular structure of the SemEval datasets, it would be interesting to have an error analysis of the results. I know that the space is quite limited, but a graph with a brief explanation of the main insights would be helpful for the reader. --------------------- I thank the authors for their effort in preparing the rebuttal. After reading their reply, I confirm the score given earlier.
Review 2 (by anonymous reviewer)
(RELEVANCE TO ESWC) Ontology-driven hence appropriate for ESWC. (NOVELTY OF THE PROPOSED SOLUTION) The novelty of the solution is fair compared to authors previous work. The ontology have been improved with more specific knowledge and a two stage approach is introduced instead of a combination of the knowledge and machine learning based approaches. (CORRECTNESS AND COMPLETENESS OF THE PROPOSED SOLUTION) The authors proved that the proposed solution achieves a good performance in the SemEval evaluation. (EVALUATION OF THE STATE-OF-THE-ART) The related work section is quite comprehensive and the authors compare previous approaches with their proposed solution. (DEMONSTRATION AND DISCUSSION OF THE PROPERTIES OF THE PROPOSED APPROACH) The different aspects are presented in a comprehensive manner, the algorithm is well explained. (REPRODUCIBILITY AND GENERALITY OF THE EXPERIMENTAL STUDY) The experimental setup being an open evaluation, the experimental study is reproducible. (OVERALL SCORE) Summary of the Paper The paper describes an ontology-based sentiment analysis method. It is based on domain knowledge described in an ontology and when not applicable, it relies on a bag of words model. The method is proven to have a high performance on the 2015 and 2016 editions of the SemEval Aspect-Based Sentiment Analysis task. Strong Points (SPs) The paper is easy to read and well written and structured. The algorithm is clearly explained with simple and useful examples. The evaluation section is exhaustive well commented with clear conclusions. Weak Points (WPs) The usefulness of two stage method (which is one of the main contribution of the paper) compared to the previously proposed hybrid solution is not clear. As mentioned by the authors, the performance does not depend on the amount of available training data but it is still outperformed by hybrid approaches even with small sets of training data. The main contribution is the new classification of the sentiment words which even if it is shown to be effective stays in my opinion relatively low in terms of novelty of the approach.
Review 3 (by Víctor Rodríguez Doncel)
(RELEVANCE TO ESWC) I have carefully gone through the bullet points at:
Metareview by Valentina Presutti
The paper proposes an ontology-driven approach to aspect based sentiment analysis. It is well written and introduce novelty with respect to previous work and related literature. It is appreciated that the ontology and the source code are available, helping to support reproducibility. The authors are recommended to follow reviewers' suggestions to bring the paper at its best form and in particular to include a fair account on the comparison with SemEval.