The result
The 600 games were 300 deals, each played twice with colours swapped, at 4 seconds a move on 4 threads. As in jqe's match against the last AB-JChess, our referee drew any threefold repetition and any 60 moves by each side without a capture. When a face-down piece was taken, it told both engines what it was; on Mistboard only the player who took it finds out.
| AB-JChess | Won-drawn-lost | Score |
|---|---|---|
| All 600 games | 306-67-227 | 57% |
| With Red | 172-34-94 | 63% |
| With Black | 134-33-133 | 50% |
Almost all of AB-JChess's edge came with Red. Over the match it comes to about +46 Elo, with a 95% range of +23 to +69. Scored by Mistboard's own repetition rule it would be about +41, as explained below.
Five games, read by both engines
How to read these games
Each board opens on a move where the engines disagreed, played by the side that went on to lose or draw. Wherever the two would have played different moves, a blue arrow shows AB-JChess's choice and an orange one jqe's. The graph under the board is the winning side's chance by each engine at match settings, AB-JChess in teal and jqe in orange, and the comments in the move list say where the game turned and what each engine expected. A die beside a reveal grades its luck from what both players could see; the study has all six games.
AB-JChess wins game 1: one soldier push loses the endgame
Every piece is face up and the game is level. jqe (Red) pushes a soldier, where AB-JChess would give its horse for Black's advisor. AB-JChess sees at once that the push lets Black's cannon swing over and attack Red's general; jqe sees it only eight moves later. Black wins on move 103, when Red has no legal move. Why did jqe push? Not for lack of time: with 16 times the search it still picks the push and still calls the game a dead draw. It misjudged the position rather than missing a move. AB-JChess, with its chase rule off and the same extra search, still sees the push lose.
AB-JChess wins game 2: a reveal decides it
Black's chariot has raided Red's back rank. jqe brings it home; AB-JChess would leave it there. Two later moves decide it: on move 35 Red turns over a soldier and both engines call the game even, and on move 36 Red takes a hidden piece and both have Red winning. Red wins on move 66. Both engines saw those two moves the same way, so this game does not tell us which was right about the chariot.
AB-JChess wins game 3: jqe's cannon starts a losing run of trades
Red is ahead after its cannon took a hidden piece on move 14. jqe's cannon move starts a run of trades: Black gives a soldier and its horse for an elephant and an advisor. AB-JChess thinks the trades are bad for Black; jqe thinks they are fine and only agrees two moves later. Red wins on move 72. Three hidden pieces turned over in a row helped change jqe's mind, so this example is weaker than game 1.
jqe wins game 4: two reveals go wrong for AB-JChess
Many pieces are still hidden, and jqe already thinks Black is better. AB-JChess makes a quiet cannon move where jqe would turn over a hidden piece, and neither engine can see past a reveal, so we can't say who was right. Luck decides it: AB-JChess turns over a soldier on move 19, then a chariot that Black's cannon takes at once. Black wins on move 30.
Game 5 is drawn: AB-JChess has it won and repeats
Red is far ahead for most of the game, but Black slowly comes back. After Red turns over a soldier on move 50, AB-JChess thinks Black is winning. Then Red's chariot and Black's elephant move back and forth until the same position comes up three times, and the game is drawn. AB-JChess did not see the repetition coming, because in the match each engine was given only the current position, not the moves before it.
Game one's eight moves is the long end; in the other four the lag was two or three. Across the match, counting from the move where each engine's own scores gave the winner 70% for good, the two were within a move of each other in 232 of AB-JChess's 304 wins, and where one was later it was jqe in 119 and AB-JChess in 72. In jqe's 227 wins each was later about as often, AB-JChess in 76 and jqe in 75.
How their play differs
jqe overrates a small edge
When jqe scored its own position as a small edge, it won less often from there than AB-JChess did from the same score.
| Small edge (+50 to +150), moves 6 to 30 | jqe | AB-JChess |
|---|---|---|
| Went on to score | 46% | 64% |
| Positions | 592 | 595 |
jqe takes more face-down pieces, win or lose
Both take more in the games they win, so the table splits by result.
| Face-down pieces taken a game | jqe | AB-JChess |
|---|---|---|
| In games it won | 2.2 | 1.8 |
| In games it drew | 2.2 | 1.4 |
| In games it lost | 1.8 | 1.4 |
It costs jqe nothing we can measure. In close positions, its winning chance one move after a face-down capture fell 1.7 points on average and AB-JChess's 0.7, a gap smaller than the noise in the sample.
What the match tells us
Half the deals decided themselves
Each deal was played twice with the colours swapped. In 147 of the 300 deals the same colour won both games, Red in 93 and Black in 54, so whichever engine had that side, the deal decided it. AB-JChess's edge comes from the rest: it won both games of a deal 56 times, and jqe did 34 times.
AB-JChess's edge is judgement, not depth
jqe was the later of the two to see the result in AB-JChess's wins, while in jqe's wins neither engine was. In game 1 jqe still called a lost endgame a dead draw with 16 times the search, and when it scored itself a small edge it went on to score less than AB-JChess did from the same score. Both point the same way: jqe's mistakes are in how it values quiet positions and trades, not in how far it looks. Neither engine searches past a reveal, so turning a piece over stays a guess for both, as in game 4.
Most draws were not draws by Mistboard's rules
Our referee drew every threefold repetition. Mistboard does not: a side that gives check on every move of the repetition loses. In 42 of the 67 draws one side did that, and in 40 of them both engines already scored the other side as winning, so checking forever was the losing side's way out. Scored by Mistboard's rules, AB-JChess won 323, drew 25 and lost 252, about +41 Elo with a 95% range of +17 to +66. The +46 above keeps the referee's rules, the same ones as jqe's match against the last AB-JChess, so the two matches can be compared.
Of the 25 games that stay drawn, 9 were won for one side by both engines' scores and repeated anyway, 8 of them for AB-JChess. That is the blindness game 5 showed: in the match each engine saw only the current position, so it could not see a repetition coming. On Mistboard the bots are sent the moves since the last capture or reveal, so they do see it.
The engines also bring their own rule for a chase, a repetition in which one side keeps attacking the same piece, and neither matches Mistboard's. jqe calls a side lost when every one of its moves in the repetition chases the same piece. AB-JChess calls a side lost when it chased on any move and the other side never did.
Red's chariot then attacks the horse after every move it makes. By jqe's rule Red loses, and jqe gave Black 72% or more to the end; by AB-JChess's rule the horse chases back, so it scored a draw. Mistboard draws a chase, so jqe was counting on a rule the game is not played under. Step through the board for both engines' rules searched on and off.
A smaller network, a faster search
The new release is rebuilt on jqe's code, which Devin O'Keefe published under the MIT licence, with a network of its own. That network is 0.6 MB, where the last release's was 144 MB and jqe's is 1.5 MB. A small network is quick to run, so the new release looks at about 65 times as many positions a second as the last one, a little more than jqe.
In 4 seconds each engine searches about 22 million positions. To see whether either was short of time, each one searched its own moves from 20 of the games again with twice that. AB-JChess would have changed its move 20% of the time and jqe 18%, and the change was worth 0.07 points of winning chance a move to each. A change worth 5 points or more came up once in 754 moves for AB-JChess and once in 750 for jqe. More time would help both a little and neither more than the other.
Game review
Game review for jieqi runs on AB-JChess, so new reviews use the new release. On our server it searches about 47 times as many positions a second as the old one, so a review looks further in the same time.
The score in a review is the engine's own number, and each network counts on its own scale. The old network's scale was steep: +1.1 already meant a 75% chance. The new one counts on about the same scale as jqe, where 75% is +4.1. So the same position now shows a bigger number, and we fitted the new network's winning chances on 9,776 positions from this match. Old reviews keep the old numbers, so compare an old review with a new one by winning chance, not by the score.
Thanks
AB-JChess is open source under the GPL-3.0 licence at github.com/lxsgx23/AB-JChess. The network is downloaded from the authors' own release each time we build the site, never re-hosted. Thank you, Huorongrong and Laoxu (Kouza). Laoxu trains AB-JChess's networks and, for this release, tested four ways of searching a reveal and kept the best; he and Huorongrong worked out together the small, fast network it plays with. And thank you, Devin O'Keefe, for the code the new release stands on.