Paper 88 (Research track)

Leveraging the Dynamic Changes from Items to Improve Recommendation

Author(s): Zongze Jin, Yun Zhang, Weimin Mu, Weiping Wang, Hai Jin

Full text: submitted version

Abstract: User-generated reviews contain rich information, which has been ignored by most of recommender systems. Recently, some recommender systems using reviews with deep learning techniques have demonstrated that they can potentially alleviate the sparsity problem and improve the quality of recommendation. However, they only consider the dynamic interests from users but ignoring the changed properties of items. In this paper, we present a deep model which can capture not only the common users’ behaviors, the changed users’ interests and fundamental item properties, but also the changed properties of items. Experimental results conducted on a variety of datasets demonstrate that our model significantly outperforms all baseline recommender systems.

Keywords: Recommender System; Dynamic Item Reviews; Deep Learning

Decision: reject

Review 1 (by anonymous reviewer)

(RELEVANCE TO ESWC) The usage of reviews to provide further details to a recommender system is in line with the topic of this conference.
(NOVELTY OF THE PROPOSED SOLUTION) The idea is interesting, however there are some conceptual errors (see below).
(CORRECTNESS AND COMPLETENESS OF THE PROPOSED SOLUTION) See below.
(EVALUATION OF THE STATE-OF-THE-ART) Related word provides the necessary tools to understand the topic and challenge addressed in the paper.
(DEMONSTRATION AND DISCUSSION OF THE PROPERTIES OF THE PROPOSED APPROACH) Clear demonstration and discussion of the proposed approach.
(REPRODUCIBILITY AND GENERALITY OF THE EXPERIMENTAL STUDY) 2 datasets are publicly available. 1 was created by authors and enough details about its creation have been reported, however the final dataset has not been shared. 
Clear evaluation metrics.
The implementation is sufficiently described however there are some lacks in the parameter optimization what makes the full reproducibility at risk.
(OVERALL SCORE) The authors propose a recommender system based on a deep learning architecture designed to exploit textual reviews about the items. While in previous works researchers have only investigated the temporal evolution of users' preferences, authors claim that it is necessary to also consider the dynamic changes that could occur to the items. The proposed architecture combines a CNN and an LSTM network for modeling the users with another CNN and LSTM network for modeling the items. This solution was validated with an empirical experiment and it systematically achieved the best results.
Even if the idea is very interesting and the usage of reviews is in line with the topic of this conference, the paper presents several issues that make its publication unadvisable.
The paper contains some conceptual errors: for example, it is not correct that a high number of reviews implies a high sparsity, but the opposite is true, as, for example, explained in the Recommender Systems Handbook by Ricci et al.
The general structure and several sentences are almost identical to the ones of "Joint Deep Modeling of Users and Items Using Reviews for Recommendation" by Lei Zheng et al. For example, the first paragraph of the section describing the experiment is the same. Furthermore, the first sentences describing the approach differ only for the name of the technique. Many other examples can be found while reading the paper. Obviously, authors should not copy sentences word-by-word from published papers and they should at least try to reformulate them, as this behavior is unethical.
Finally, there are many grammatical errors and typos, which make the reading very difficult. Authors should also carefully proofread their work.


Review 2 (by anonymous reviewer)

(RELEVANCE TO ESWC) The authors talk the "semantics" in a different way as ESWC community.
(NOVELTY OF THE PROPOSED SOLUTION) It is a novel application of deep learning, but the authors have not developed new deep learning techniques, or any new semantic techniques.
(CORRECTNESS AND COMPLETENESS OF THE PROPOSED SOLUTION) The solutions seems is correct.
(EVALUATION OF THE STATE-OF-THE-ART) It is a novel use but the deep learning method is developed by others.
(DEMONSTRATION AND DISCUSSION OF THE PROPERTIES OF THE PROPOSED APPROACH) It is well discussed.
(REPRODUCIBILITY AND GENERALITY OF THE EXPERIMENTAL STUDY) Not sure but the experiments seem correct.
(OVERALL SCORE) It is an interesting paper to apply deep learning for studying user-generated reviews for recommendation. It is not in the scope of the ESWC because: 
It's not clear what do authors mean "semantic structure of the reviews" or "semantics of the reviews."  It is not the same formal semantics (i.e., logic) in Semantic Web context.  
Figure 3 is important, but too small, hard to read.


Review 3 (by Tommaso Di Noia)

(RELEVANCE TO ESWC) This paper provides an interesting discussion in the field of users and items modeling, taking into account temporal changes to which both of them are subject.
(NOVELTY OF THE PROPOSED SOLUTION) This work presents a novel idea, which is that of items properties changes leading to a definition of a new way to improve recommendation quality.
(CORRECTNESS AND COMPLETENESS OF THE PROPOSED SOLUTION) Main details about the proposed solution are well explained.
(EVALUATION OF THE STATE-OF-THE-ART) The effectiveness of the proposed approach is confirmed by a large number of baselines to which it is compared; appropriate strategies have been taken into account.
(DEMONSTRATION AND DISCUSSION OF THE PROPERTIES OF THE PROPOSED APPROACH) Several experiments involving different configurations are carried out, in order to better discuss gathered results.
(REPRODUCIBILITY AND GENERALITY OF THE EXPERIMENTAL STUDY) Exploited datasets are publicly available and the described experiments are well reproducible.
(OVERALL SCORE) ---------------------------Summary of the paper--------------------------------
The authors build a novel recommender system based on users' text reviews and deep learning techniques. The main idea is that not only users' behavior changes over time, but also items properties. Hence, two different models for users and items are built separately. They leverage CNN and LSTM networks in order to extract both static and dynamic features from users' reviews, exploiting the resulting vector to compute recommendation. Experiments are carried out on three datasets (Yelp16, Skytrax, and Trip) and achieved results are compared with several state-of-the-art methods.
---------------------------Strong Points--------------------------------------
- Novel idea: in some domains, items properties may change over time; considering this aspect may lead to better recommendation quality.
- Exploiting users' reviews to model both users and items dynamic features.
- The authors analyze several aspects of the proposed model investigating the effect of alternative techniques (such as GRU instead of LSTM and other types of word embedding in reviews pre-processing phase).
---------------------------Weak Points----------------------------------------
- A larger explanation about word embedding should be provided.
- The lack of a short description about RNNs, CNNs, and their capabilities.
- The choice of evaluation metrics should be motivated.
---------------------------Questions to the Authors---------------------------
1. Why did you choose those evaluation metrics rather than others?
2. Which evaluation protocol is used?
---------------------------AFTER REBUTTAL---------------------------
As pointed out by another reviewer, the general structure and several sentences are almost identical to the ones of "Joint Deep Modeling of Users and Items Using Reviews for Recommendation" by Lei Zheng et al. Given This, the paper cannot be accepted for publication at ESWC.


Metareview by Stefan Dietze

As pointed out by the reviewers, this submission has significant overlap with another one not authored (yet cited) by this work: https://arxiv.org/abs/1701.04783. After inspecting both papers, the overlap apparently concerns (a) the overall idea (modeling items and user through separate NNs), (b) large parts of the technical decisions about the NN architecture, (c) the experimental setup and (d) entire paragraphs which are copy/pasted without any reference to the original work. Even though some of the experimental data differs, due to the fact that the authors replaced some of the used datasets, the overlap is striking, in particular given that this is a machine learning paper where the contribution resides in the overall idea and technical choices about the architecture. For these reasons, this submission violates ethical scientific practices.


Share on

Leave a Reply

Your email address will not be published. Required fields are marked *