***NAACL-HLT 2019*** Accept rate: Overall: 423/1733 = 24.4% Long: 281/1067 = 26.3%, Short: 142/666 = 21.3% PAPER: 133 TITLE: Personalized Neural Embeddings for Collaborative Filtering with Text AUTHORS: Guangneng Hu ============================================================================ REVIEWER #1 ============================================================================ What is this paper about, and what contributions does it make? --------------------------------------------------------------------------- Problem/Question: Authors start from the observation that "MF and neural CF exploit the user-item behavior interactions only and hence suffer from the data sparsity and cold-start issues." and claim that one possible way to alleviate the issue is to consider the textual information associated with reviews. The paper then addresses the problem of incorporating textual features in CF models by means of a Memory Network storing relationships between users and text of reviews entered. Contributions (list at least two): 1. A Novel hybrid architecture for CF mixing a traditional Neural CF approach with a Memory Network 2. Couldn't find a second one. --------------------------------------------------------------------------- What strengths does this paper have? --------------------------------------------------------------------------- Strengths (list at least two): 1. The paper is extremely easy to read and clear. Experiments are very nicely done and comparisons with the state of the art is extensive. 2. The memory network component really improves an already good baseline (MLP) 3. Experiments are deep and thorough and give new insights on the use of a Memory Network for improving Collaborative Filtering methods. --------------------------------------------------------------------------- What weaknesses does this paper have? --------------------------------------------------------------------------- Weaknesses (list at least two): 1. The novelty of the approach is somewhat limited as MemoryNet has been used already in the past although with a different focus [Ebesu et al., 2018] 2. In the paper I was looking for some kind of intuition behind the use of a MemoryNet instead of other structures for exploiting textual features from users 3. Authors justify the adoption of textual features to counteract cold-start and/or sparse profiles. I can't find any experiments supporting the fact that their architecture is good in these kind of cases. --------------------------------------------------------------------------- --------------------------------------------------------------------------- Reviewer's Scores --------------------------------------------------------------------------- Overall Score (1-6): 5 Readability (1-5): 5 Additional Comments (Optional) --------------------------------------------------------------------------- Authors start from the observation that "MF and neural CF exploit the user-item behavior interactions only and hence suffer from the data sparsity and cold-start issues." and claim that one possible way to alleviate the issue is to consider the textual information associated with reviews. It's not clear how this feature will help users who have never reviewed an item, who I believe are the majority of users. The main contribution is the proposal of a neural framework to exploit interaction relational data and content information seamlessly. Personalized Neural Embedding (PNE) exploits jointly semantic representations learnt from reviews with behavior representations learnt from interactions of users with sold items. Authors claim this technique to be new and while in one of the papers they refer [Ebesu et al., 2018] authors are combining memory network and CF they are not using memory network to store information about the textual content of reviews. Basically the main idea is to combine MLP (He et al., 2017) with a memory network, the output of MLP and the output of the average attentive scores for users and items from the memory network are concatenated and fed into logistic regression. It would be interesting to They evaluate the model on two real-world data.The public Amazon product dataset and a dataset from the Cheetah Mobile news company. Results show that they can achieve improvements across all the datasets (with the sole exception of hit-ration in the case of the Amazon dataset w.r.t. the best baseline - MLP). It is not clear if those improvements are all statistically significant therefore it would be good to add a statistical significance analysis. On the evaluation side, I'd like to point out that basically MLP is like your model but without the MemoryNet component. It might be nice to point this out in your paper because it will add a further insight, which is the fact that MemoryNet, in fact, is beneficial to solving the task you address. --------------------------------------------------------------------------- ============================================================================ REVIEWER #2 ============================================================================ What is this paper about, and what contributions does it make? --------------------------------------------------------------------------- Problem/Question: This paper develops a personalized neural embedding framework to exploit both user-item relational interactions and words in unstructured text. Embeddings of users, items, and words are learned jointly, and user preferences are predicted on items based on these learned representations. To model the behavior factor, this paper adopts a neural collaborative filtering approach, which learns the user-item nonlinear interaction relationships using a neural network. To model the semantic factor, this paper adopts a memory network to match word semantics with the specific user via the attention mechanism inherent in the memory module. Contributions (list at least two): 1. The proposed model fuses semantic representations learned from unstructured text with behavior representation learned from user item interactions jointly for estimation on preferences of users to items. 2. The proposed algorithm was tested on two real-world datasets and it outperformed state-of-the-art models and it learns meaningful embeddings. --------------------------------------------------------------------------- What strengths does this paper have? --------------------------------------------------------------------------- Strengths (list at least two): 1. The proposed model fuses semantic representations learned from unstructured text with behavior representation learned from user item interactions jointly for estimation on preferences of users to items. 2. The proposed algorithm was tested on two real-world datasets and it outperformed state-of-the-art models and it learns meaningful embeddings. --------------------------------------------------------------------------- What weaknesses does this paper have? --------------------------------------------------------------------------- Weaknesses (list at least two): Some of the words in the visualization of the word embeddings were either too small or not clearly visible. Perhaps, the authors could show a smaller subset of the visualization and maybe use different colors for the words in the visualization. There was no section discussing the related work. --------------------------------------------------------------------------- --------------------------------------------------------------------------- Reviewer's Scores --------------------------------------------------------------------------- Overall Score (1-6): 5 Readability (1-5): 4 ============================================================================ REVIEWER #3 ============================================================================ What is this paper about, and what contributions does it make? --------------------------------------------------------------------------- Problem/Question: To augment standard Collaborative Filtering with Text, this work proposes the use of Personalized Neural Textual Embeddings, so to better exploit the user-item relational data and address the common user-item data sparsity issue for recommender systems. Contributions (list at least two): 1. This work presents a novel personalized neural embedding (PNE) model to jointly learn representations of users, items, and textual words. PNE is designed to captures nonlinear user-item interaction relationships and matches word semantics with user preferences. 2. On two real-world datasets, PNE shows better performance than state-of-the-art. This work also shows that PNE learns meaningful word embeddings by visualization. --------------------------------------------------------------------------- What strengths does this paper have? --------------------------------------------------------------------------- Strengths (list at least two): 1. This work proposes method that can utilize texts which provides independent and diverse information sources essential for recommendation beyond user-item interaction data. This methods hence can alleviate the data sparsity issue. 2. This work conducts extensive experiments and provided strong results. The details are convincing. --------------------------------------------------------------------------- What weaknesses does this paper have? --------------------------------------------------------------------------- Weaknesses (list at least two): 1. Put URL addresses in Section 3 to footnotes. 2. It would be easier to read if all equations can be put as independent lines, and also numbered properly. 3. Augmenting texts is not a highly novel idea. "As far as we know, PNE is the first neural embedding model that integrates relational interactions data with unstructured text by bridging neural CF and memory networks. PNE can learn meaningful word embeddings and reveal novel findings." I prefer to tune down this claim a bit: e.g. 'reveal novel findings'? adding textual information helps recommendation system, which in fact is not a too much 'novel finding' (as similar idea has been discovered extensively previously). Please rephrase it. 4. More ablation study --------------------------------------------------------------------------- --------------------------------------------------------------------------- Reviewer's Scores --------------------------------------------------------------------------- Overall Score (1-6): 4 Readability (1-5): 4