KDD Reviews 周天烁 2018年5月9日

Outline Manuscript & Reviews Manuscript Reviews

Reviews Review 1 Review 2 Review 3 Review 4 Novelty : 3: (Incremental) 2: (Standard) 4: (Some novelty) Quality : 3: (Fair) 4: (Good) 2: (Weak) Presentation : 1: (No) 2: (Yes) Overall evaluation : -2: (I am not championing but if there is a champion then I am fine accepting) -4: (I believe this should be rejected)

Motivation Schema-simple Graph meta-path

Schema-rich Graph Motivation

Automatically select Meta-Path in relavance search AutoMP Automatically select Meta-Path in relavance search

Strengths Interesting problem of using meta-paths selection for relevance search; This paper propose an interesting problem formulation. This paper distinguished the schema-rich and schema-simple HIN, and demonstrate different performance comparison on each case. The topic of automated meta-path selection is interesting and significant to HIN community. The introduction clearly explains the problem of user-guided relevance search with concrete examples. The paper is well motivated and provides good background of the problem.

The motivation is not convincing The motivation is not well justified, why study such a problem? The authors said it has important applications in web search and product recommendation, but the situation of 'user-guided' search in HIN seems rarely happen in web search or product recommendation. And the examples illustrated in Fig. 2 are artificial. One attempts to know the collaborators of Jiawei Han could simply search name in the DBLP and view the column of 'refine by co-author', while one want to know outstanding researcher like Jiawei Han in related fields would use the keyword 'data mining' to search the DBLP database. It is a little bit counterintuitive to image a query processing where the user would give some sample results to guide his/her search.

AutoMP Meta-Path Selection Weight Learning Significance of Meta-Path heuristic forest Weight Learning generation of high-quality negative examples SVM

Significance of Meta-Path example entities representative entities Scoreing function

Greedy Tree query example entities representative entities

Generate negative examples + ? P1 P2 ? query ? … + ? ? Pn ?

Soft margin SVM

Strengths A possible, reasonable solution for the problem. This paper proposes a new approach to users-guided relevance search called AutoMP that can automatically identify a set of significant meta-paths that can best characterize the user's intent indicated by the example entities. AutoMP can generate high-quality negative examples based on the identified meta-paths. The model is capable of discovering and exploiting the semantics of relevance indicated by example entities, and well suits the relevance search task.

Weaknesses Technical merit of the proposed meta-path selection (especially meta- path forests) is not clear; Missing important references and related work is out of date; Some model choices are not well justified. Theoretical contributions are minimal. Some components in the proposed method are not explained clearly. How to build meta-path forest should be explained more clearly.

Experiment Dataset Baseline DBLP DBpedia YAGO UGSS[12’KDD] Relsim [16’SIAM] Relsim+ AutoMP- DMP[15’WWW]

Weaknesses Weak experiments. The number of datasets is limited, and the results are not stable. Experiments are not sound. The experiments are not convincing. Some related baselines are not included. Only one schema-rich network is used. There is no case study to clearly illustrate the work.

Suggestions / Conclusion Clarify the concept novelty. The authors may need to address more in depth on the problems stated above for the AutoMP model. It seems that the performances of AutoMP rely heavily on high-quality negative samples. Meanwhile, the meta-path forest doesn't work well. Improve experiments. Provide more convincing use cases of the proposed problem setting. Do experiments on more datasets (e.g., facebook). It’s strongly recommend that the authors expand the experimental evaluations. You should do more experiments on schema-rich HINs to validate the validity of AutoMP.

THANKS