In-game advertisements have become a significant source of revenue for developers of freemium mobile games. Despite this, we are not aware of any published exploratory study looking into the effect of these advertisements on individual, user-level behavior. We consult existing literature on the psychology of advertising and interruption, and propose several metrics that could be used to measure user-level retention. Using a unique dataset representing 21 free-to-play mobile games, we create a simple model which attempts to determine the effect of advertisements on retention. We find only a weak relationship between advertisements and retention. More importantly we find that game-specific effects dominate all advertising effects, yielding important results for game development and design decisions. While the results presented have limitations, they provide a starting point for future research on individual level analysis.