I built a xiangqi puzzle miner

The games

They come from ElephantChess, which publishes its own site’s games as anonymised monthly dumps under GPL-3.0. Amateur games, which matters: strong players do not blunder often enough to be a supply.

A run freezes its game list before any engine time is spent, sampled across ratings, time controls, results and lengths so it does not turn out to be all blitz. Nothing is added to a run once it starts.

The algorithm

Two passes: a cheap one over every position of every game, and an expensive one over the few that survive it.

The cheap pass replays a game and stops at every position after ply 8, asking Pikafish for its top two moves at 60,000 nodes. That is roughly depth 10 to 14, and it is shallow on purpose, because it runs everywhere.

A position becomes a candidate when the move actually played loses at least 250 centipawns against the engine’s best, and the position it leaves behind is winning by at least 250 centipawns for the other side. A blunder that leaves the game equal is not a puzzle. There is nothing to find.

A centipawn is a hundredth of a soldier, the unit engines use for material. Xiangqi has no pawn, so the name comes from chess along with the scale. The values this site uses put a horse or a cannon at 450 and a chariot at 900, which makes a 250-centipawn swing about half a horse.

Two filters keep that pass honest. Positions already decided by 800 centipawns are skipped, because winning a won game harder is not a tactic. And no game gives up more than three candidates, so one collapse cannot flood the corpus with variations on itself.

The expensive pass takes each candidate back to the engine at depth 20 and 600,000 nodes, ten times the budget, handed over as a bare FEN with no move history. Same position, no context, so the engine cannot lean on the search it just did.

Then the line is built one solver move at a time, and every move has to be uniquely best on its own. That is what separates a puzzle from a plausible sequence. A principal variation is one line the engine liked from one search. It says nothing about whether move three was forced, and a solver who finds a different move three and is told they are wrong has been lied to.

Uniqueness is not a centipawn gap. Two moves 50 centipawns apart are both fine, and demanding one punishes a solver for choosing correctly. What makes a move the answer is that every alternative is wrong: it gives the win away, or it wins materially less. The test is a hand-tuned cascade rather than anything principled, and it fails closed, so anything it cannot separate is thrown away.

The code

The cheap pass, condensed. One detail halves it: judging a move needs the position’s value before and after, and both are already there if the scans stay in order, because the move played is worth the negation of the next position’s score. One search per position, not two.

// packages/game/src/puzzles-xiangqi-mining.ts (condensed)

// scans[i] is the engine's best score at the position BEFORE move i, from the
// point of view of whoever is to move there. That one array is enough to
// judge every move in the game: the value of the move played from
// position i is -scans[i + 1], because position i + 1 is the same position
// scored by the opponent. One search per position, not two.

for (let ply = minPly; ply < moveCount; ply += 1) {
  const pre = scans[ply];       // the best that was available
  const post = scans[ply + 1];  // what they left behind, opponent's view
  if (pre === null || post === null) continue;

  if (Math.abs(pre) >= decidedCp) continue;  // already decided: no tactic
  if (post < winCp) continue;                // solver must end up winning

  const playedCp = -post;                    // the move, in their own terms
  const swing = pre - playedCp;
  if (swing < swingCp) continue;             // a mistake, but a small one

  candidates.push({ ply, swingCp: swing, preBestCp: pre, postBestCp: post });
}

And the gate. Every branch that returns false here is a way a real blunder fails to be a puzzle.

// packages/game/src/puzzles-xiangqi-mining.ts (condensed)

const winRate = (cp) => 1 / (1 + 10 ** (-cp / 400));

// Is this solver move THE answer, or merely a good one? Every branch that
// returns unique:false is a reason a real blunder failed to become a puzzle.

function classifySolverMove(best, second) {
  if (!best) return { unique: false, reason: 'missing-best' };

  // Mate saturates both centipawns and win%, so mates get their own rule:
  // unique only when this is the strictly fastest forced mate.
  if (mates(best)) {
    if (!second || !mates(second))
      return { unique: true, reason: 'fastest-mate' };
    return best.mate < second.mate
      ? { unique: true, reason: 'fastest-mate' }
      : { unique: false, reason: 'mate-not-unique' };
  }

  if (winRate(best.scoreCp) < 0.8)
    return { unique: false, reason: 'best-not-winning' };
  if (!second) return { unique: true, reason: 'only-move' };
  if (mates(second)) return { unique: false, reason: 'runner-up-mates' };

  const gapCp = best.scoreCp - second.scoreCp;
  if (gapCp < 200) return { unique: false, reason: 'near-tie' };

  // The runner-up is wrong if it gives the win away outright...
  if (winRate(second.scoreCp) <= 0.6)
    return { unique: true, reason: 'runner-up-loses-win' };
  // ...or if it still wins, but wins a whole piece less.
  if (gapCp >= 250) return { unique: true, reason: 'material-gap' };

  return { unique: false, reason: 'alternative-still-good' };
}

What it keeps

Two thirds of the puzzles open with a move that captures nothing. If you hunt for tactics by scanning the captures first, which is what most of us do, you are looking at the wrong third of the board most of the time. Step through this one: the chariot goes the length of the board and takes nothing on the way.

Only about a tenth involve giving material away. Sacrifices are the tactics people remember, so I had assumed they would be a larger slice. In real games between real players, the winning move is usually just a move. Here is one of the tenth, and the solver is the one behind: a horse and a cannon down before it starts.

Not every puzzle ends in mate. About 40% end with the solver simply winning, and those are the ones a mate-shaped intuition misses. This one opens with a move that takes nothing, hands a soldier back, and collects an advisor and both horses for it.

And one that takes nothing for four plies. Red is 150 centipawns down here and the engine calls the position level.

What it throws out

The rejects define a puzzle better than the keeps do, because each one is a real blunder that failed for exactly one reason.

Near-tie is the biggest, about a third. The player had a winning move and so did something else. Both work, so there is no answer to check against and no puzzle, even though the blunder was real and the position was winning.

Too short is another third, and it gives the clearest example in the corpus of what the miner is for. Below is a position it rejected. Black is to move, the engine scores it as a forced mate against a second-best line of +1407, and exactly one of Black’s twenty legal moves does it.

The other way to fail is to have too many answers rather than too few. Red is to move below and the horse on e8 mates two different ways, c9 or g9. The stepper plays one of them. Either wins, so there is nothing to check a solver against, and the candidate is thrown out.

Promised mate not reached is the narrow one, and it is not a rule against non-mate puzzles. It fires only when the engine returned a mate score, so the line promised a mate, and replaying it inside the seven-ply cap never got there. The promise could not be checked, so the candidate goes. A position with an ordinary winning evaluation never enters that branch at all, and ships as one of the winning-advantage puzzles above.

What a puzzle turns out to be

Most winning positions have several winning moves, and that is what disqualifies them: a third of everything found died on that alone. A puzzle is a position with one answer, deep enough that finding it takes work, and stable enough that a stronger engine still agrees an hour later.

That is a much narrower thing than a mistake. Nine out of ten mistakes do not qualify.

One caveat I would rather say than hide: the gate has never been checked against a human. Its four thresholds came from reading rejected positions, not from measuring whether the puzzles they admit are any good, and the win-rate curve they act on is inherited from chess. Solve rates and reveal rates are recorded per puzzle, so the data to grade it exists.