Determining the amount of sequestered carbon in soils and monitoring soil health in farmlands is an important climate change problem. Motivated by the lack of scalable and inexpensive techniques for in-situ soil health monitoring, we focus on low-voltage nuclear magnetic resonance (NMR) spectroscopy as a promising new approach and develop a reinforcement learning technique to modulate NMR pulses for rapid atomic abundance assessment of soils. Our preliminary results derived using Monte Carlo sampling and parallelized OpenAI Gym training show the promise of our RL-based approach.