TurnWord's Word Bot, Benchmarked: 2,433 Games, One Opener, Zero Failures

We built TurnWord Word Bot to solve 5-letter word puzzles, step by step, with no access to the hidden answer and no peeking at an answer key. (The architecture itself isn't limited to five letters — word length is a parameter throughout the engine, and 4-, 6-, and 7-letter configurations already exist — but 5-letter is the only length with a trained scoring model and a shipped app today, so that's what this piece benchmarks.) This post isn't a generic "how Wordle bots work" explainer. It's a benchmark of the actual system running at turnword.com right now, with the methodology shown so the numbers can be checked rather than taken on faith.

Two word counts show up throughout this piece: Word Bot can type any of 14,823 legal 5-letter words as a guess, but only 2,433 of those are ever the actual answer. The benchmark below tests all 2,433; the guesses along the way are free to come from the larger 14,823. Both numbers are a snapshot — the answer list especially gets curated over time — but nothing here hinges on the exact count; the mechanics and results hold the same way whether that number is 2,433 or a few dozen either side of it.

The headline result: we played all 2,433 words in our current answer list through Word Bot's own API, once each, always opening with the bot's own top-ranked first guess. Word Bot guesses under the same free-play permissions real Wordle uses — any known word is a legal guess at any point — but its scoring model keeps it picking a word consistent with every green, yellow, and gray seen so far almost every turn anyway, the same discipline a careful human would apply. The one deliberate exception is the probe, covered later in this piece: a guess pulled from outside that consistent candidate list, used only when the gating logic decides it's actually worth the trade. That free-play permission isn't incidental — pin the bot to always guessing strictly consistent words and a handful of same-family boards (BOUND/FOUND/HOUND/MOUND/POUND/ROUND/SOUND/WOUND, differing only in their first letter, is the textbook case) become mathematically unsolvable within six guesses, because eliminating one candidate at a time never arrives in time. Free play is what makes the probe possible in the first place.

Zero failures across the entire answer list, at an average under 3.7 guesses. Both of those numbers matter, and getting both at once is the harder problem — as the rest of this post shows.

Here's the full pipeline behind every one of those guesses, from the dictionary down to a single recommendation, before we get into why guess three onward is the hard part:

Filtering has to be exactly right first

Before any strategy matters, the bot has to correctly compute which words are still possible given the board. Get this wrong and no amount of cleverness in guess selection saves you.

The case that breaks naive implementations is a repeated letter. Guess SPEED against the answer ABATE and the real result is:

S → absent

P → absent
E → present (yellow)
E → absent
D → absent



Read the second, later E as "no E in the word" — a common mistake — and the puzzle becomes unsolvable by construction: "contains an E" (from the yellow) and "contains no E" (from the naive reading of the gray) are direct opposites, so nothing can satisfy both at once, for any dictionary. Zero isn't a result here — it's a contradiction dressed up as one. The correct reading is narrower but very different: the target has exactly one E, and it isn't at either position already tried. We checked this directly against our answer list — the correct interpretation leaves 1,231 real candidates (ABATE, ACUTE, AFIRE, AFORE, AGATE, AGAVE, and more), not zero.

Word Bot's constraint engine resolves this with two rules, not one:

• A letter with no yellow or green anywhere on the board is fully eliminated.
• A letter that already has a yellow or green, and then comes back gray, is capped — present, but not at this position and not more times than already confirmed.

That distinction is worth checking for yourself in any Wordle tool: pick a board with a repeated letter and see whether the candidate count collapses to zero. If it does, the filter has this bug.

Why raw information isn't the whole strategy

Once filtering is solid, the question becomes which word to guess next, and Word Bot answers it differently depending on how far into the game it is. On guess 2, it ranks candidates by Shannon entropy — how evenly a guess is expected to split the field, averaged across every possible feedback pattern. That addresses the flat spot in ranking by raw word frequency alone (favoring only "is this likely to be the answer," never "what would this teach me if it isn't"), but entropy is an average-case measure: it doesn't know that winning this turn is worth more than its bare probability suggests, because winning ends the game outright instead of merely shrinking the field.

That distinction sharpens in the endgame, and it isn't entropy doing the work there — from guess 3 on, Word Bot switches to ranking by worst case: for each candidate, the largest group of survivors any single feedback pattern could still leave, smallest worst case first. A guess drawn from outside the remaining candidates entirely — a "probe" — can sometimes beat every real candidate on that measure, by testing letters no candidate word can test at once. Taking a probe purely because it wins on worst-case split can trade a small amount of average-case speed for a meaningfully better floor — but that trade isn't automatic or guaranteed to pay off on any single board, which is why Word Bot doesn't apply it with a fixed rule; it's a call the trained scoring model makes case by case. Across the full distribution, that's how the benchmark above lands on a strong average and zero observed failures, instead of trading one for the other.

A real probe word, from an actual game

Here's that decision in action, not as theory. We played the answer BATCH through Word Bot's live API starting from SLATE — one of its five proven openers, not the top-ranked one, just to show the mechanic isn't tied to a single starting word:

Guess 3 is the interesting one. Between SLATE and TAINT, the board already knows S, L, E, I, and N are all absent, A sits at position 2, and there's exactly one T, in neither of the two spots TAINT tried it. CLIMB restates two of those — its L and I are redundant — but C, M, and B are all genuinely untested, and CLIMB cannot be the answer itself (it fails the T constraint outright). Word Bot guesses it anyway, purely for what those three new letters reveal: that single guess narrows 22 candidates down to 1. It's a genuine probe, chosen deliberately, not a mistake — the API marks it explicitly, with a coverage score of 9/9: however the remaining field could have split on this guess, this move was still going to reach the answer in time.

Skip the probe and Word Bot still finds BATCH, but it costs two full guesses. Replaying the identical board at Strictest difficulty (candidates only, no probes allowed) gives TAINT → MATCH → WATCH → HATCH → BATCH: six guesses instead of four, spent grinding through the "-ATCH" family one real candidate at a time (22 → 7 → 6 → 5) instead of testing three distinguishing letters in a single move.

That two-guess gap isn't guaranteed, though — it depends on the opener, not just the target. Play the exact same answer, BATCH, starting from Word Bot's top-ranked opener PLATE instead, and a probe still fires (WHICH this time — itself a nice second confirmation of the capped-not-eliminated rule above, since WHICH has a repeated H and the feedback correctly shows one occurrence absent, one correct), but it no longer saves anything: probed and unprobed both land on six guesses, just by different routes — PLATE → TAINT → WHICH → CATCH → MATCH → BATCH versus PLATE → TAINT → MATCH → WATCH → HATCH → BATCH. Same target, same mechanism, two different outcomes depending on where the game started. That's the trade-off from above, concretely: a real average-case win, not a guaranteed one on any single board.

Note: probes are currently in internal testing and haven't shipped to turnword.com yet — expect them live this fall. Every number in this piece, including the zero-failure benchmark, was measured against this same in-testing build.

How TurnWord Word Bot is actually trained

The scoring model behind these decisions isn't trained on human game logs — those are noisy, and there usually aren't enough of them. It's trained on games the bot plays against itself, in a loop that starts from a hand-built heuristic rather than any learned model or human data:

1. Bootstrap with a heuristic player. Before any learned model exists, a hand-tuned scorer — weighing information gain, word frequency, letter frequency, and positional coverage — plays tens of thousands of games against real targets. Every turn becomes a training example: the candidate word, the game state, and a quality score for having guessed it.
2. Oversample the hard cases. Real benchmark data flags which targets the bot historically struggles with. Those get resampled up to 5× more often, so training time is spent disproportionately on actual weak spots rather than games the bot already wins easily.
3. Train a model. The scoring network takes a candidate word, the feedback history, and a feature vector spanning game state, database statistics, word-intrinsic properties, and strategic signals — and predicts one number: how good is this guess, right now.
4. Self-play. Once a model exists, it plays against itself to generate the next round of data, with deliberate randomness mixed in so it keeps encountering situations its current policy would otherwise avoid.
5. Retrain, and repeat.

The label each guess receives is the part that matters most, and it isn't decided in the same turn it's made. Every guess starts with a score based on how much it shrank the candidate pool — but after the game ends, every guess in it gets revised: a won game adds a bonus scaled by how quickly it finished; a lost game subtracts a penalty that grows the later the mistake came, with the final couple of turns forced toward a bad label outright. A guess that looked clever in isolation — a wide, information-rich split — gets marked down after the fact if the game it was part of still lost. That's the direct answer to the probe-word trade-off above: instead of scoring a guess by what it teaches you this turn, the training signal is shaped by what actually happened by the end of that specific game.

Stress test: forcing the worst possible opener

The benchmark above uses Word Bot's own top-ranked opener, PLATE. To check how much of the result depends on starting well, we forced a different first guess: FUZZY, the single lowest-information real answer-eligible word in our list, measured the same way as every other entropy figure in this piece — Shannon entropy of the feedback distribution over all 2,433 answer-eligible targets, weighted uniformly. FUZZY scores 2.31 bits against PLATE's 5.52 — the kind of first guess nobody optimizing for information would ever choose.

Both openers were run against the same 300 highest-frequency targets, so the comparison isn't PLATE's full sweep against a smaller, different sample of FUZZY games:

A deliberately terrible first guess cost about 0.7 guesses on average on this sample — and still didn't cause a single failure. That's the adaptive layer doing its job: most of the damage from a bad start gets absorbed by the turns that follow rather than compounding through them.

Try it yourself

TurnWord Word Bot itself is live today: the repeated-letter case, the "-ATCH" games, the entropy-driven second guess — check any of it at turnword.com, no account, no ads, alongside the rest of the suite (Word Solver, Advanced Finder, Quick Finder). The probe strategy specifically is still on our internal testing build, planned for release this fall; once it ships, we'll publish the benchmark methodology alongside it so the probe-specific numbers in this piece can be reproduced the same way.