Compression of Strings with Approximate Repeats

Proceedings Of The Sixth International Conference On Intelligent Systems For Molecular Biology

Volume

Proceedings Of The Sixth International Conference On Intelligent Systems For Molecular Biology

Contents

We describe a model for strings of characters that is loosely based on the Lempel Ziv model with the addition that a repeated substring can be an approximate match to the original substring; this is close to the situation of DNA, for example. Typically there are many explanations for a given string under the model, some optimal and many suboptimal. Rather than commit to one optimal explanation, we sum the probabilities over all explanations under the model because this gives the probability of the data under the model. The model has a small number of parameters and these can be estimated from the given string by an expectation-maximization (EM) algorithm. Each iteration of the EM algorithm takes O(n^2) time and a few iterations are typically sufficient. O(n^2) complexity is impractical for strings of more than a few tens of thousands of characters and a faster approximation algorithm is also given. The model is further extended to include approximate reverse complementary repeats when analyzing DNA strings. Tests include the recovery of parameter estimates from known sources and applications to real DNA strings. Keywords: pattern discovery, repeats, sequence analysis, hidden Markov model, DNA, data compression.