My goal is to create AI systems that learn from few samples to robustly make good decisions. This is a crucial part of human intelligence that enables people to adapt and even thrive in changing environments. While scientific knowledge can help guide interventions, there remains a key need to quickly cut through the space of decision policies to find effective strategies to support people. Reinforcement learning (RL), the subfield of artificial intelligence focused on agents that learn through experience to make high utility choices, is a powerful framework for addressing these challenges. However, most recent efforts in RL have focused on settings where good or perfect simulators exist, and it is cheap and feasible to collect millions of trials. In contrast, my work focuses on the many societal applications – treating patients with advanced liver cancer or training people to be computer technicians – which both lack good simulators and are inherently data limited. I create new reinforcement learning algorithms and conduct theoretical analysis to understand what is possible, all inspired and motivated by my applications in education and healthcare.