A Method for Measuring Sentence Similarity and Its Applicationto Conversational Agents

May 2004

Proceedings of the Seventeenth International Florida Artificial Intelligence Research Society Conference (FLAIRS 2004)

Volume

Proceedings of the Seventeenth International Florida Artificial Intelligence Research Society Conference (FLAIRS 2004)

All Papers

This paper presents a novel algorithm for computing similarity between very short texts ofsentence length. It will introduce a method that takes account of not only semantic information but also word order information implied in the sentences. Firstly, semantic similarity between two sentences is derived from information from a structured lexical database and from corpusstatistics. Secondly, word order similarity is computed from the position of word appearance in the sentence. Finally, sentence similarity is computed as a combination of semantic similarity and word order similarity. The proposed algorithmis applied to a real world domain of conversational agents. Experimental results demonstrated that the proposed algorithm reduces the scripter’s effort to devise rule base for conversational agent.