 Autonomous exploration of materials design space is hindered by its high-dimensionality and a scarcity of data. In this work, we present AlphaFlow, a self-driven lab guided by reinforcement learning that enables accelerated discovery and optimization of multi-step canastries. This article was authored by Amanda A. Vogue, Robert W. Epps, Daniel T. Yanomoto, and others.