Solving Wordle Using Actor-Critic Reinforcement Learning
subject: actor-critic methods; curriculum learning; neuro-symbolic AI; reinforcement learning; word games; wordle
description: The popular word-game Wordle poses a difficult sequential decision-making problem with enormous discrete action spaces and partial feedback. In order to solve Wordle as efficiently as possible, this work explores the use of actor-critic reinforcement learning techniques. We develop two actor-critic variations, Vanilla Actor-Critic (AC) and Advantage Actor-Critic (A2C), and formulate Wordle as a Markov Decision Process. Through curriculum training on increasingly larger vocabulary, our method filters out invalid actions and guides learning by combining neural networks and symbolic reasoning. While the AC agent has 42.35 % success with an average guess of 4.85, the A2C agent has a 46.05 % success rate averaging 5.31 guesses per successful game. We show that, especially in the worst-case situations, batch-based A2C performs more robustly than stepbased AC. By effectively scaling from tiny vocabularies (50 words) to the entire Wordle lexicon (14,855 words), our neuro-symbolic technique demonstrates the efficacy of curriculum learning for challenging word games. 2025 IEEE.
creator: Bhat, Soujanya M; Ramasamy, Gobi; Syam Mohan, E.
source: 2025 9th International Conference on Computational System and Information Technology for Sustainable Solutions, CSITSS 2025;
publisher: Institute of Electrical and Electronics Engineers Inc.
date: 01-01-2025
rights: Restricted Access; Hardcopy may be available in the library
relation: ISBN: 979-833158894-6;
language: English
type: Conference paper
coverage: Bhat S.M., Christ University, Department of Computer Science, Bangalore, India; Ramasamy G., Christ University, Department of Computer Science, Bangalore, India; Syam Mohan E., Christ University, Department of Computer Science, Bangalore, India