The rapid growth of large scale event datasets with timestamps has given rise to the dynamically evolving multi-relational knowledge graphs. Temporal reasoning over such data brings on many challenges and is still not well understood. Most real-world knowledge graphs are characterized by a long-tail relation frequency distribution where a significant fraction of relations occurs only a handful of times. This observation has given rise to the recent interest in low-shot learning methods that are able to generalize from only a few examples. The existing approaches, however, are tailored to static knowledge graphs and not easily generalized to temporal settings, where data scarcity poses even bigger problems, due to the occurrence of new, previously unseen relations. The goal of my doctoral research is to introduce new approaches for learning meaningful representation that capture the dynamics of temporal knowledge graphs while tackling various existing challenges such as data scarcity.