Qing Li, S. K. Candan, Yan Qi
Search engines present fix-sized passages from documents ranked by relevance against the query. In this paper, we present and compare novel, language-model based methods for extracting variable length document snippets by real-time processing of documents using the query issued by the user. With this extra level of information, the returned snippets are considerably more informative. Unlike previous work in passage retrieval which rely on searching relevant segments for filtering of preoccupied passages, we focus on query-informed segmentation to extract context-aware relevant snippets with variable length. In particular, we show that, when informed through an appropriate relevance language model, curvature analysis and Hidden Markov model (HMM) based content segmentation techniques can help extract relevant document snippets.
Subjects: 13. Natural Language Processing; 13.1 Discourse
Submitted: Apr 11, 2008