This is an extended abstract of a paper to be published at ICAPS 2021. We study online Multi-Agent Path Finding (MAPF), where new agents are constantly revealed over time and all agents must find collision-free paths to their given goal locations. We generalize existing complexity results of (offline) MAPF to online MAPF. We classify online MAPF algorithms into different categories. We present several complexity and competitiveness results for online MAPF and its algorithms, which provides theoretical insights into the effectiveness of using MAPF algorithms in an online setting for the first time.