Solving Wordle Using Actor-Critic Reinforcement Learning

Title

Solving Wordle Using Actor-Critic Reinforcement Learning

Creator

Bhat, Soujanya M; Ramasamy, Gobi; Syam Mohan, E.

Description

The popular word-game Wordle poses a difficult sequential decision-making problem with enormous discrete action spaces and partial feedback. In order to solve Wordle as efficiently as possible, this work explores the use of actor-critic reinforcement learning techniques. We develop two actor-critic variations, Vanilla Actor-Critic (AC) and Advantage Actor-Critic (A2C), and formulate Wordle as a Markov Decision Process. Through curriculum training on increasingly larger vocabulary, our method filters out invalid actions and guides learning by combining neural networks and symbolic reasoning. While the AC agent has 42.35 % success with an average guess of 4.85, the A2C agent has a 46.05 % success rate averaging 5.31 guesses per successful game. We show that, especially in the worst-case situations, batch-based A2C performs more robustly than stepbased AC. By effectively scaling from tiny vocabularies (50 words) to the entire Wordle lexicon (14,855 words), our neuro-symbolic technique demonstrates the efficacy of curriculum learning for challenging word games. 2025 IEEE.

Source

2025 9th International Conference on Computational System and Information Technology for Sustainable Solutions, CSITSS 2025;

Date

01-01-2025

Publisher

Institute of Electrical and Electronics Engineers Inc.

Subject

actor-critic methods; curriculum learning; neuro-symbolic AI; reinforcement learning; word games; wordle

Coverage

Bhat S.M., Christ University, Department of Computer Science, Bangalore, India; Ramasamy G., Christ University, Department of Computer Science, Bangalore, India; Syam Mohan E., Christ University, Department of Computer Science, Bangalore, India

Rights

Restricted Access; Hardcopy may be available in the library

Relation

ISBN: 979-833158894-6;

Format

online

Language

English

Type

Conference paper

Identifier

https://doi.org/10.1109/CSITSS67709.2025.11294192

https://www.scopus.com/pages/publications/105031416529?origin=resultslist