The 6th Computer Olympiad: Computer-Games Workshop (Maastricht, 2001)

Advances of AND/OR-tree Search Advances of AND/OR-tree Search

Algorithms in Shogi Mating SearchAlgorithms in Shogi Mating Search

Hiroyuki Iida and Makoto Sakuta

Department of Computer Science

Shizuoka University, Japan

OverviewOverview

• Tsume-shogi (mating search) problem is Tsume-shogi (mating search) problem is a good application for AND/OR-tree a good application for AND/OR-tree search algorithmssearch algorithms

• Algorithms for solving tsume-shogiAlgorithms for solving tsume-shogi

– ID, AO*, PN search, PN*, PDS and DF-PNID, AO*, PN search, PN*, PDS and DF-PN

– Equivalent relations between these Equivalent relations between these algorithmsalgorithms

• Tsume-shogi in actual shogi endgameTsume-shogi in actual shogi endgame

– Find a relatively long-step winning Find a relatively long-step winning sequencesequence

Main rules of tsume-shogiMain rules of tsume-shogi

• Two agents: attacker and defenderTwo agents: attacker and defender

• Attacker has to continue the check move Attacker has to continue the check move sequences.sequences.

• Attacker must mate in the shortest sequence.Attacker must mate in the shortest sequence.

• Defender must select a move which makes the Defender must select a move which makes the mating sequence as long as possible. mating sequence as long as possible.

• The length of the solution sequence is not given The length of the solution sequence is not given in advance.in advance.

• See also [Seo, Iida and Uiterwijk, 2001] See also [Seo, Iida and Uiterwijk, 2001]

Why tsume-shogi is a good Why tsume-shogi is a good application for AND/OR-tree application for AND/OR-tree

search?search?

• Tsume-shogi has an enormous search space, Tsume-shogi has an enormous search space, so simple brute-force search is not adequate.so simple brute-force search is not adequate.

• A tsume-shogi problem has a unique solution.A tsume-shogi problem has a unique solution.

• There are various types of problems, There are various types of problems, including those very hard to solve and those including those very hard to solve and those having a very long solution sequence.having a very long solution sequence.

• Many high-performance tsume-shogi solving Many high-performance tsume-shogi solving programs have been developed, so the programs have been developed, so the performance of a program can be measured performance of a program can be measured and compared.and compared.

Examples of solving tsume-Examples of solving tsume-shogishogi (1)(1)

• An endgame position from GM’s game. An endgame position from GM’s game. (mate in 17 plies, relatively not so long)(mate in 17 plies, relatively not so long)

– ID solved in 22.6 seconds. ID solved in 22.6 seconds.

– PN solved in 7.93 seconds. PN solved in 7.93 seconds.

– PDS solved in 6.11 seconds.PDS solved in 6.11 seconds.

– PN* solved in 4.87 seconds.PN* solved in 4.87 seconds.

• See the detail (machine spec, size of See the detail (machine spec, size of transposition table, etc.) in [Sakuta and Iida, transposition table, etc.) in [Sakuta and Iida, ACG9].ACG9].

Examples of solving tsume-shogi Examples of solving tsume-shogi (2)(2)

• An endgame position from GM’s game. An endgame position from GM’s game. (mate in 37 plies, relatively long in actual games)(mate in 37 plies, relatively long in actual games)

– ID was unable to solve. ID was unable to solve.

– PN was unable to solve. PN was unable to solve.

– PN* solved in 1000 seconds.PN* solved in 1000 seconds.

– PDS solved in 757 seconds.PDS solved in 757 seconds.

• See the detail (machine spec, size of See the detail (machine spec, size of transposition table, etc.) in [Sakuta and Iida, transposition table, etc.) in [Sakuta and Iida, ACG9].ACG9].

Why proof-number search?Why proof-number search?

• Best-firstBest-first search algorithms such as proof-number (PN) search algorithms such as proof-number (PN) search work well when solving endgame positions in search work well when solving endgame positions in chess-like games. (also other games: GO, LOA, etc.)chess-like games. (also other games: GO, LOA, etc.)

• However, it is hard, even for PN search, to solve an However, it is hard, even for PN search, to solve an endgame problem with very long sequences because endgame problem with very long sequences because of its inherent memory limits.of its inherent memory limits.

• Recently, innovative depth-first search algorithms Recently, innovative depth-first search algorithms based on proof/disproof numbers were developed based on proof/disproof numbers were developed (PN*, PDS and DF-PN) for solving tsume-shogi (PN*, PDS and DF-PN) for solving tsume-shogi problems, among which one with 1525 steps.problems, among which one with 1525 steps.

• These depth-first algorithms behave like best-first These depth-first algorithms behave like best-first search. search.

AND/OR-tree searchAND/OR-tree search

• Best-first searchBest-first search

AO* (Nilsson) and its variantsAO* (Nilsson) and its variants

Minimax treeCN-search (McAllester, 1988)(Schaeffer, 1990)conspiracy number

PN-search (Allis,1994)PN-search (Allis,1994)

proof numberproof numberdisproof number disproof number

A* for OR Graph

PN-search (Allis, 1994)PN-search (Allis, 1994)• PN (proof number) search is a best-first AND/OR tree-search algorithm.PN (proof number) search is a best-first AND/OR tree-search algorithm.

• Inspired by conspiracy-number search algorithm (McAllester, 1988).Inspired by conspiracy-number search algorithm (McAllester, 1988).

• Before starting the search, a search goal is defined. E.g., try to mate. Before starting the search, a search goal is defined. E.g., try to mate. The evaluation of a node returns one of three values: The evaluation of a node returns one of three values: true, false, true, false, or or unknownunknown. The evaluation is seen from the point of view of the player to . The evaluation is seen from the point of view of the player to move in the root position. The value move in the root position. The value truetrue indicates that the player to indicates that the player to move in the root position can achieve the goal, while move in the root position can achieve the goal, while falsefalse indicates indicates that the goal is unreachable. that the goal is unreachable.

• A node is A node is provedproved if its value has been established to be if its value has been established to be truetrue, whereas , whereas the node is the node is disproveddisproved if its value has been determined to be if its value has been determined to be falsefalse. .

• A node is A node is solvedsolved as soon as it has been proved or disproved. as soon as it has been proved or disproved.

• A tree is solved (proved or disprove) if its root is solved.A tree is solved (proved or disprove) if its root is solved.

• The goal of PN search is to solve a tree.The goal of PN search is to solve a tree.

PN-search (continue)PN-search (continue)

• Has high ability for solving AND/OR trees.Has high ability for solving AND/OR trees.

• However, it expands the whole search tree in However, it expands the whole search tree in working memory and require much memory.working memory and require much memory.

• This means that there is a severe memory This means that there is a severe memory restriction for hard problems.restriction for hard problems.

• The detail of PN search [Allis et al., 1994]The detail of PN search [Allis et al., 1994]

• PN^2 search (Breuker et al., 1999)PN^2 search (Breuker et al., 1999)

• Would it be possible to transform a best-first PN Would it be possible to transform a best-first PN search into a depth-first search?search into a depth-first search?

PN* (Seo, 1995) PN* (Seo, 1995)

• is recursive iterative-deepening depth-first is recursive iterative-deepening depth-first search algorithm using proof numbers as a search algorithm using proof numbers as a criterion to develop the frontier nodes.criterion to develop the frontier nodes.

• PN* starts searching by setting the proof-PN* starts searching by setting the proof-number threshold to 1 and expands the root number threshold to 1 and expands the root node. If the proof number of the node under node. If the proof number of the node under consideration exceeds the threshold, PN* stops consideration exceeds the threshold, PN* stops node expansion at this node. When all frontier node expansion at this node. When all frontier nodes have proof numbers exceeding the proof-nodes have proof numbers exceeding the proof-number threshold, the search tree is discarded, number threshold, the search tree is discarded, the threshold is incremented and a new search the threshold is incremented and a new search starts. starts.

• This iterative searching is enhanced by storing This iterative searching is enhanced by storing the expanded nodes and their properties in the the expanded nodes and their properties in the transposition table. transposition table.

PN* (continue) PN* (continue)

• As the threshold is getting larger, node As the threshold is getting larger, node expansion proceeds in such a way that nodes expansion proceeds in such a way that nodes having relatively small proof numbers are having relatively small proof numbers are expanded first, like in best-first search, without expanded first, like in best-first search, without the disadvantage of the memory requirement.the disadvantage of the memory requirement.

• PN* uses the method of PN* uses the method of recursive iterative recursive iterative deepeningdeepening. At an AND node the threshold values . At an AND node the threshold values assigned to the child OR nodes are iteratively assigned to the child OR nodes are iteratively increased from 1 to their current maximum.increased from 1 to their current maximum.

• Enhanced by dynamic evaluation, efficient Enhanced by dynamic evaluation, efficient successor ordering and pruning by dependency successor ordering and pruning by dependency relations. relations.

• The detail of PN* [Seo, Iida and Uiterwijk, 2001]The detail of PN* [Seo, Iida and Uiterwijk, 2001]

Recursive iterative deepeningRecursive iterative deepening

threshold=2

threshold=3

threshold=4

recursive iterative deepeningrecursive iterative deepening

threshold=2

threshold=3

threshold=4

depth=1

depth=2 or 3

depth=3 or 5

depth=4 or 7

depth=5 or 9

best-first search (e.g. pn-search)

iterative-deepening depth-first search

frontier nodes

iterative deepening at root nodeiterative deepening at root node

Equivalence (1): AO* and PN*Equivalence (1): AO* and PN* (Nagai, (Nagai, 1999)1999)

• PN* (depth-first search) algorithm behaves the PN* (depth-first search) algorithm behaves the same as AO* (best-first search) in the meaning same as AO* (best-first search) in the meaning that PN* always expands a that PN* always expands a most-proving nodemost-proving node..

• Most proving node (in the context of AO*) is a Most proving node (in the context of AO*) is a leaf node selected by tracing from the root in leaf node selected by tracing from the root in the following way.the following way.

• For each OR node, trace the child with For each OR node, trace the child with minimum proof number.minimum proof number.

• For each AND node, trace any child with non-For each AND node, trace any child with non-zero proof number. zero proof number.

Tsume-shogi solver: STsume-shogi solver: SEOEO

• based on the PN* algorithmbased on the PN* algorithm

• excellent management of transposition excellent management of transposition tabletable

• uses Dominance Relation of positionsuses Dominance Relation of positions

• omits the useless moves of dropping piecesomits the useless moves of dropping pieces

• other domain-specific featuresother domain-specific features

• solved “Microcosmos”, the problem with solved “Microcosmos”, the problem with the longest steps, i.e., 1525 plies (1997) the longest steps, i.e., 1525 plies (1997)

PDS PDS (Nagai,1998)(Nagai,1998)

• PDS (PDS (Proof-number and Disproof-number SearchProof-number and Disproof-number Search) is a ) is a straight extension of PN*, using both proof straight extension of PN*, using both proof numbers and disproof numbers. PDS uses two numbers and disproof numbers. PDS uses two thresholds in searching, the proof-number thresholds in searching, the proof-number threshold and the disproof-number threshold. threshold and the disproof-number threshold.

• If the proof number or the disproof number If the proof number or the disproof number exceeds the threshold at a certain node, PDS exceeds the threshold at a certain node, PDS stops node expansion at this node. When PDS stops node expansion at this node. When PDS fails the root node expansion, it increases the fails the root node expansion, it increases the either threshold value and restarts the searching. either threshold value and restarts the searching.

• Note that the basic concept of PDS differs from Note that the basic concept of PDS differs from PN search, i.e., asymptotically equivalent to PN PN search, i.e., asymptotically equivalent to PN search.search.

DF-PN DF-PN (Nagai,1999)(Nagai,1999)

• Depth-first version of PN-searchDepth-first version of PN-search

• DF-PN first sets, at the root node, the DF-PN first sets, at the root node, the thresholds of both proof number and thresholds of both proof number and disproof number to a certain large value disproof number to a certain large value ((∞). ∞). Then, the threshold values are Then, the threshold values are distributed among the descendant nodes.distributed among the descendant nodes.

• AAt every node, recursive iterative t every node, recursive iterative deepening is performed (cf. only OR deepening is performed (cf. only OR nodes in PN*)nodes in PN*)

Equivalence (2): PN and DF-PN (Nagai, Equivalence (2): PN and DF-PN (Nagai, 1999)1999)

• DF-PN (depth-first search) algorithm behaves DF-PN (depth-first search) algorithm behaves the same as PN (best-first search) in the the same as PN (best-first search) in the meaning that DF-PN always expands a meaning that DF-PN always expands a most-most-proving nodeproving node..

• Most proving node (in the context of PN) is a Most proving node (in the context of PN) is a leaf node selected by tracing from the root in leaf node selected by tracing from the root in the following way.the following way.

• FFor each OR node, trace the child with or each OR node, trace the child with minimum proof number.minimum proof number.

• FFor each AND node, trace the child with or each AND node, trace the child with minimum disproof number.minimum disproof number.

Tsume-shogi solver: NTsume-shogi solver: NAGAIAGAI

• based on the DF-PN algorithmbased on the DF-PN algorithm

• efficient use of transposition tableefficient use of transposition table– SmallTreeGC and SmallTreeReplacement (Nagai, SmallTreeGC and SmallTreeReplacement (Nagai,

99)99)

• pruning by dependency relations (positions and pruning by dependency relations (positions and proof/disproof numbers) proof/disproof numbers) – More efficient than the solver More efficient than the solver SSEOEO

• GHI problem (cf. Breuker et al., CG98)GHI problem (cf. Breuker et al., CG98)

• Proof pieces (Seo, 99) and disproof piecesProof pieces (Seo, 99) and disproof pieces

• detecting a DAG to avoid the double counting detecting a DAG to avoid the double counting of proof numbersof proof numbers

Performance of PN*, PDS and PN Performance of PN*, PDS and PN search (Sakuta and Iida, ACG9, search (Sakuta and Iida, ACG9,

1999)1999)• Experiments in two domains: Experiments in two domains:

– tsume-shogi (derived from GM’s games), andtsume-shogi (derived from GM’s games), and

– 6x6 othello (positions on PV)6x6 othello (positions on PV)

• Comparison shows that Comparison shows that – In both, PDS outperforms PN*.In both, PDS outperforms PN*.

– In othello, PN search variants do not show In othello, PN search variants do not show remarkable advantage over depth-first search.remarkable advantage over depth-first search.

– In shogi, PN search variants show remarkable In shogi, PN search variants show remarkable advantage over depth-first search. advantage over depth-first search.

• Forcing move sequences and sudden Forcing move sequences and sudden termination.termination.

• Disproving ability is important as well as Disproving ability is important as well as proving. proving.

Disproving ability compared in tsume-Disproving ability compared in tsume-shogishogi

PN* PDS PN-search iterative deepening

ex1p ○ 0.6

○ 0.9

○ 0.3

○ 28.5

ex1d × ○ 1.5

○ 12.1

×

ex2p ○ 0.7

○ 0.6

○ 0.6

○ 0.6

ex2d × ○ 1.0

○ 4.1

×

Four algorithms (equivalent)Four algorithms (equivalent)

Proof number Proof number and disproof number

Best-first search AO* PN search

Depth-first search PN* Df-pn

More detail comparisonMore detail comparisonIterative Iterative deepenindeepeningg

PNPN PN*PN* PDSPDS DF-PNDF-PN

behavioubehaviourr

depth-depth-firstfirst

best-firstbest-first depth-depth-firstfirst

depth-depth-firstfirst

depth-depth-firstfirst

transpositiotransposition tablen table

yesyes basically basically nono

indispensabindispensablele

indispensabindispensablele

indispensabindispensablele

conspiracy conspiracy numbernumber

nono proof &proof &

disproofdisproof

proofproof proof &proof &

disproofdisproof

proof &proof &

disproofdisproofiterative iterative deepeningdeepening

depthdepth nono proofproof

thresholthresholdd

proof &proof &

disproofdisproof

proof &proof &

disproofdisproof

recursive recursive iterative iterative deepeningdeepening

nono nono at OR at OR nodesnodes

at at AND/ORAND/OR nodesnodes

at at AND/ORAND/OR nodesnodes

iteration at iteration at the root the root nodenode

yesyes yesyes proofproof

thresholthresholdd

proof &proof &

disproofdisproof

nono

Advantages of depth-first-type Advantages of depth-first-type algorithm algorithm

• If a best-first algorithm and a depth-first If a best-first algorithm and a depth-first algorithm behave in the same way, it is algorithm behave in the same way, it is certainly better to adopt the depth-first one.certainly better to adopt the depth-first one.– IDA* and A*, MT-SSS* and SSS*, and IDA* and A*, MT-SSS* and SSS*, and

– MTD(f) for minimax trees [Plaat et al., 1996] MTD(f) for minimax trees [Plaat et al., 1996]

• Many enhancements can be used:Many enhancements can be used:– History heuristics, transposition table, iterative History heuristics, transposition table, iterative

deepening, and so on.deepening, and so on.

• Depth-first algorithm uses less memory spaceDepth-first algorithm uses less memory space– Search speed is often faster in practiceSearch speed is often faster in practice

Summary of experimental Summary of experimental performanceperformance

• PN* solved most hard problems, PN* solved most hard problems, among which one with 1525 steps. among which one with 1525 steps. But, itBut, it was unable to solve 10 hard was unable to solve 10 hard problems with long steps over 300. problems with long steps over 300.

• PDS basically outperforms PN* in PDS basically outperforms PN* in solving long-step problems.solving long-step problems.

• DF-PN have solved all hard-problems.DF-PN have solved all hard-problems.

Application to actual endgames in Application to actual endgames in shogishogi

• Advances of AND/OR tree search Advances of AND/OR tree search algorithms enable a computer to:algorithms enable a computer to:

– Find a mate under tournament time Find a mate under tournament time constraints, at interior nodes (close to the constraints, at interior nodes (close to the root). root).

– Reuse the outcome of mating search in the Reuse the outcome of mating search in the successive searches.successive searches.

– Find a brink-mate (forcing mate threat)Find a brink-mate (forcing mate threat)

– Play a high-level endgame (mating race).Play a high-level endgame (mating race).

ConclusionsConclusions

• Depth-first searches using proof/disproof Depth-first searches using proof/disproof numbers have recently been developed.numbers have recently been developed.– Basically depth-first search but behaves in best-Basically depth-first search but behaves in best-

first manner like proof number search (PN first manner like proof number search (PN search).search).

– Memory requirement is much less than best-first. Memory requirement is much less than best-first.

• DF-PN have solved all hard-problems of DF-PN have solved all hard-problems of tsume-shogi with long steps over 300. tsume-shogi with long steps over 300.

• These algorithms are now used in shogi These algorithms are now used in shogi playing programs.playing programs.– High-level mating race in the endgame.High-level mating race in the endgame.