Human intelligence has long inspired new benchmarks for research in artificial intelligence. However, recently, research in machine learning and AI has influenced research on children’s learning. In particular, Bayesian frameworks capture hallmarks of children’s causal reasoning: given causally ambiguous evidence, prior beliefs and data interact. However, we suggest that the rational frameworks that support rapid, accurate causal learning can actually lead children to generate and maintain incorrect beliefs. In this paper we present three studies demonstrating these surprising misunderstandings in children and show how these errors in fact reflect sophisticated inferences.