Solving Wordle Using Actor-Critic Reinforcement Learning
actor-critic methods; curriculum learning; neuro-symbolic AI; reinforcement learning; word games; wordle
The popular word-game Wordle poses a difficult sequential decision-making problem with enormous discrete action spaces and partial feedback. In order to solve Wordle as efficiently as possible, this work explores the use of actor-critic reinforcement learning techniques. We develop two actor-critic variations, Vanilla Actor-Critic (AC) and Advantage Actor-Critic (A2C), and formulate Wordle as a Markov Decision Process. Through curriculum training on increasingly larger vocabulary, our method filters out invalid actions and guides learning by combining neural networks and symbolic reasoning. While the AC agent has 42.35 % success with an average guess of 4.85, the A2C agent has a 46.05 % success rate averaging 5.31 guesses per successful game. We show that, especially in the worst-case situations, batch-based A2C performs more robustly than stepbased AC. By effectively scaling from tiny vocabularies (50 words) to the entire Wordle lexicon (14,855 words), our neuro-symbolic technique demonstrates the efficacy of curriculum learning for challenging word games. 2025 IEEE.
Bhat, Soujanya M; Ramasamy, Gobi; Syam Mohan, E.
2025 9th International Conference on Computational System and Information Technology for Sustainable Solutions, CSITSS 2025;
Institute of Electrical and Electronics Engineers Inc.
Restricted Access; Hardcopy may be available in the library
https://doi.org/10.1109/CSITSS67709.2025.11294192 https://www.scopus.com/pages/publications/105031416529?origin=resultslist
Bhat S.M., Christ University, Department of Computer Science, Bangalore, India; Ramasamy G., Christ University, Department of Computer Science, Bangalore, India; Syam Mohan E., Christ University, Department of Computer Science, Bangalore, India