Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
BQPJSON
Carleton CoffrinLos Alamos National Laboratory
Advanced Network Science Initiative
and Friends
LA-UR-17-28428
My Background and Interests
• Trained as a computer scientist
My Background and Interests
• Trained as a computer scientist• Specialized in algorithms for NP-Hard
optimization problems
My Background and Interests
• Trained as a computer scientist• Specialized in algorithms for NP-Hard
optimization problems• Optimization Generalist
• Local Search (LS) • Mixed-Integer Programming (MIP) • Constraint Programming (CP) • Convex Optimization • Mixed-Integer NonLinear Programming
(MINLP)
My Background and Interests
• Trained as a computer scientist• Specialized in algorithms for NP-Hard
optimization problems• Optimization Generalist
• Local Search (LS) • Mixed-Integer Programming (MIP) • Constraint Programming (CP) • Convex Optimization • Mixed-Integer NonLinear Programming
(MINLP)• Hybrid Methods
• Large Neighborhood Search (LNS) • Heuristic Column Generation Discrete Optimization
My Day Job at Los Alamos National Laboratory
Power Network Optimization
Mixed Integer Nonlinear Programs
My Day Job at Los Alamos National Laboratory
Power Network Optimization
pij = zij(gijv2i � gijvivj cos(✓i � ✓j)� bijvivj sin(✓i � ✓j))
qij = zij(�bijv2i + bijvivj cos(✓i � ✓j)� gijvivj sin(✓i � ✓j))
zij 2 {0, 1}vi 2 (0.9, 1.1)
✓i 2 RConstants Variables
Discrete
Continuous
Unbounded
Mixed Integer Nonlinear Programs
When the D-Wave Showed Up…
• An Unconstrained Binary Quadratic Program…
min :X
i,j2Ecijbibj +
X
i2Ncibi
s.t.: bi 2 {0, 1} 8i 2 N
1
When the D-Wave Showed Up…
• An Unconstrained Binary Quadratic Program…• That’s too simple to be interesting
min :X
i,j2Ecijbibj +
X
i2Ncibi
s.t.: bi 2 {0, 1} 8i 2 N
1
When the D-Wave Showed Up…
• An Unconstrained Binary Quadratic Program…• That’s too simple to be interesting
min :X
i,j2Ecijbibj +
X
i2Ncibi
s.t.: bi 2 {0, 1} 8i 2 N
1
When the D-Wave Showed Up…
Graphics Revolution
Optimization Revolution
• An Unconstrained Binary Quadratic Program…• That’s too simple to be interesting
min :X
i,j2Ecijbibj +
X
i2Ncibi
s.t.: bi 2 {0, 1} 8i 2 N
1
My First Thought
Lets Benchmark the D-Wave
Standard Approach to Benchmarking
Solvers (i.e. Algorithms) Benchmarks (i.e. Problems)
Introduction Constraints Search Applications and Future Plans Conclusions
JaCoPJava Constraint Programming Libraray
Krzysztof Kuchcinski and Radosław Szymanek
Dept. of Computer ScienceLund University, Sweden
http://www.jacop.eu
September 16, 2013
Standard Approach to Benchmarking
Solvers (i.e. Algorithms) Benchmarks (i.e. Problems)
Introduction Constraints Search Applications and Future Plans Conclusions
JaCoPJava Constraint Programming Libraray
Krzysztof Kuchcinski and Radosław Szymanek
Dept. of Computer ScienceLund University, Sweden
http://www.jacop.eu
September 16, 2013
Standard Approach to Benchmarking
Solvers (i.e. Algorithms) Benchmarks (i.e. Problems)
MINLPLIBTSPLIB
CSPLIB
MIPLIB
QPLIB
Introduction Constraints Search Applications and Future Plans Conclusions
JaCoPJava Constraint Programming Libraray
Krzysztof Kuchcinski and Radosław Szymanek
Dept. of Computer ScienceLund University, Sweden
http://www.jacop.eu
September 16, 2013
Standard Approach to Benchmarking
Solvers (i.e. Algorithms) Benchmarks (i.e. Problems)
MINLPLIBTSPLIB
CSPLIB
MIPLIB
QPLIB
Introduction Constraints Search Applications and Future Plans Conclusions
JaCoPJava Constraint Programming Libraray
Krzysztof Kuchcinski and Radosław Szymanek
Dept. of Computer ScienceLund University, Sweden
http://www.jacop.eu
September 16, 2013
Standard Approach to Benchmarking
Solvers (i.e. Algorithms) Benchmarks (i.e. Problems)
MINLPLIBTSPLIB
CSPLIB
MIPLIB
QPLIB
Typical Benchmarking Results
SCIP
MIPLIB
Typical Benchmarking Results
SCIP
MIPLIB
Typical Benchmarking Results
SCIP
MIPLIB
?
Benchmarking the D-Wave - A Match Made in Heaven
QPLIBUnconstrained Binary Quadratic Programs Mixed Integer
Quadratically Constrained Quadratic Programs
Benchmarking the D-Wave - A Match Made in Heaven
QPLIBUnconstrained Binary Quadratic Programs Mixed Integer
Quadratically Constrained Quadratic Programs
These maybe compatible!
More Challenging than Expected
Benchmarking the D-Wave - A Match Made in Heaven
QPLIBUnconstrained Binary Quadratic Programs Mixed Integer
Quadratically Constrained Quadratic Programs
These maybe compatible!
Challenges - Problem Class
QPLIB
MI-QCQPB-QP
DW2X_SYS4C12Quite General Problem Class Quite Specific
Problem Class
Challenges - Problem Class
QPLIB
MI-QCQPB-QP
DW2X_SYS4C12Quite General Problem Class Quite Specific
Problem Class
47917
Challenges - Embedding
Source Graph Target Graphqblib_3867
Challenges - Embedding
Source Graph Target Graphqblib_3867
all hard problems failed to embedded…
FAIL
back to the drawing board
D-Wave Centric Approach to Benchmarking
or-tools
D-Wave Centric Approach to Benchmarking
or-tools
Problem Generation
D-Wave Centric Approach to Benchmarking
or-tools
Problem Generation
Test Case
D-Wave Centric Approach to Benchmarking
or-tools
Problem Generation
Test Case
D-Wave Centric Approach to Benchmarking
or-tools
Problem Generation
Test CaseNo embedding necessary maximal qbit utilization
D-Wave Centric Approach to Benchmarking
Benchmarking a quantum annealing processor with the
time-to-target metric.
James King⇤, Sheir Yarkoni, Mayssam M. Nevisi, Jeremy P. Hilton, andCatherine C. McGeoch
D-Wave Systems, Burnaby, BC
August 21, 2015
Abstract
In the evaluation of quantum annealers, metrics based on ground state success rates havetwo major drawbacks. First, evaluation requires computation time for both quantum andclassical processors that grows exponentially with problem size. This makes evaluation itselfcomputationally prohibitive. Second, results are heavily dependent on the e↵ects of analognoise on the quantum processors, which is an engineering issue that complicates the study ofthe underlying quantum annealing algorithm. We introduce a novel “time-to-target” metricwhich avoids these two issues by challenging software solvers to match the results obtained by aquantum annealer in a short amount of time. We evaluate D-Wave’s latest quantum annealer,the D-Wave 2X system, on an array of problem classes and find that it performs well on severalinput classes relative to state of the art software solvers running single-threaded on a CPU.
1 Introduction
The commercial availability of D-Wave’s quantum annealers1 in recent years [1, 2, 3] has led tomany interesting challenges in benchmarking, including the identification of suitable metrics forcomparing performance against classical solution methods. Several types of di�culties arise. First,a D-Wave computation is both quantum and analog, with nothing resembling a discrete instructionor basic operation that can be counted, as in classical benchmarking scenarios; with no commondenominator for comparison we resort to runtime measurements, which are notoriously transientand unstable. A second set of issues arises from the fact that we compare an algorithm implementedin hardware to algorithms implemented in software; standard guidelines for benchmarking computerplatforms, software, and algorithms ([4, 5, 6, 7]) do not consider this mixed scenario.
Another di�culty, addressed in this paper, arises from the fact that the algorithms of interestare heuristics for an NP-hard optimization problem. Unlike decision problems, where the algorithmreturns a solution that can be verified right or wrong, optimization problems allow heuristic solu-tions that may be better or worse, and that cannot be e�ciently verified as optimal (unless P=NP).
⇤Corresponding author, [email protected], D-Wave One, D-Wave Two, and D-Wave 2X are trademarks of D-Wave Systems Inc.
1
arX
iv:1
508.
0508
7v1
[qua
nt-p
h] 2
0 A
ug 2
015
What is the Computational Value of Finite Range Tunneling?
Vasil S. Denchev,1 Sergio Boixo,1 Sergei V. Isakov,1 Nan Ding,1 RyanBabbush,1 Vadim Smelyanskiy,1 John Martinis,2 and Hartmut Neven1
1Google Inc., Venice, CA 90291, USA2Google Inc., Santa Barbara, CA 93117, USA
(Dated: January 26, 2016)
Quantum annealing (QA) has been proposed as a quantum enhanced optimization heuristic ex-ploiting tunneling. Here, we demonstrate how finite range tunneling can provide considerable com-putational advantage. For a crafted problem designed to have tall and narrow energy barriersseparating local minima, the D-Wave 2X quantum annealer achieves significant runtime advantagesrelative to Simulated Annealing (SA). For instances with 945 variables, this results in a time-to-99%-success-probability that is ⇠ 108 times faster than SA running on a single processor core. We alsocompared physical QA with Quantum Monte Carlo (QMC), an algorithm that emulates quantumtunneling on classical processors. We observe a substantial constant overhead against physical QA:D-Wave 2X again runs up to ⇠ 108 times faster than an optimized implementation of QMC on asingle core. We note that there exist heuristic classical algorithms that can solve most instancesof Chimera structured problems in a timescale comparable to the D-Wave 2X. However, we believethat such solvers will become ine↵ective for the next generation of annealers currently being de-signed. To investigate whether finite range tunneling will also confer an advantage for problems ofpractical interest, we conduct numerical studies on binary optimization problems that cannot yetbe represented on quantum hardware. For random instances of the number partitioning problem,we find numerically that QMC, as well as other algorithms designed to simulate QA, scale betterthan SA. We discuss the implications of these findings for the design of next generation quantumannealers.
I. INTRODUCTION
Simulated annealing (SA) [1] is perhaps the mostwidely used algorithm for global optimization of pseudo-Boolean functions with little known structure. The ob-jective function for this general class of problems is
Hcl
P
(s) = �KX
k=1
NXj1...jk=1
Jj1···jksj1 · · · s
j
k
, (1)
where N is the problem size, sj
= ±1 are spin variablesand the couplings J
j1...jk are real scalars. In the physicsliterature Hcl
P
(s) is known as the Hamiltonian of a K-spinmodel. SA is a Monte Carlo algorithm designed to mimicthe thermalization dynamics of a system in contact witha slowly cooling reservoir. When the temperature is high,SA induces thermal excitations which can allow the sys-tem to escape from local minima. As the temperaturedecreases, SA drives the system towards nearby low en-ergy spin configurations.
Almost two decades ago, quantum annealing (QA) [2]was proposed as a heuristic technique for quantum en-hanced optimization. Despite substantial academic andindustrial interest [3–34], a unified understanding of thephysics of quantum annealing and its potential as an op-timization algorithm remains elusive. The appeal of QArelative to SA is due to the observation that quantummechanics allows for an additional escape route from lo-cal minima. While SA must climb over energy barriers toescape false traps, QA can penetrate these barriers with-out any increase in energy. This e↵ect is a hallmark ofquantum mechanics, known as quantum tunneling. The
standard time-dependent Hamiltonian used for QA is
H(t) = �A(t)NXj=1
�x
j
+ B(t)HP
, (2)
where HP
is written as in Eq. (1) but with the spin vari-ables s
j
replaced with �z
j
Pauli matrices acting on qubitj, and the functions A(t) and B(t) define the anneal-ing schedule parameterized in terms of time t 2 [0, T
QA
](see Fig. 7). These annealing schedules can be defined inmany di↵erent ways as long as the functions are smooth,A(0) ⌧ B(0) and A(T
QA
) � B(TQA
). At the beginningof the annealing, the transverse field magnitude A(t) islarge, and the system dynamics are dominated by quan-tum fluctuations due to tunneling (as opposed to thethermal fluctuations used in SA).
The question of whether D-Wave processors realizecomputationally relevant quantum tunneling has beena subject of substantial debate. This debate has nowbeen settled in the a�rmative with a sequence of publica-tions [6, 8, 9, 12, 13, 18, 21] demonstrating that quantumresources are present and functional in the processors. Inparticular, Refs. [35, 36] studied the performance of theD-Wave device on problems where eight [37] qubit co-tunneling events were employed in a functional mannerto reach low-lying energy solutions.
In order to investigate the computational value of fi-nite range tunneling in QA, we study the scaling of theexponential dependence of annealing time with the sizeof the tunneling domain D,
TQA
= BQA
e↵D , (3)
arX
iv:1
512.
0220
6v4
[qua
nt-p
h] 2
2 Ja
n 20
16
Quantum Annealing amid Local Ruggedness and Global Frustration
James King,⇤ Sheir Yarkoni, Jack Raymond, Isil Ozfidan, Andrew D. King,Mayssam Mohammadi Nevisi, Jeremy P. Hilton, and Catherine C. McGeoch
D-Wave Systems
(Dated: March 2, 2017)
A recent Google study [Phys. Rev. X, 6:031015 (2016)] compared a D-Wave 2X quantum process-ing unit (QPU) to two classical Monte Carlo algorithms: simulated annealing (SA) and quantumMonte Carlo (QMC). The study showed the D-Wave 2X to be up to 100 million times faster thanthe classical algorithms. The Google inputs are designed to demonstrate the value of collectivemultiqubit tunneling, a resource that is available to D-Wave QPUs but not to simulated annealing.But the computational hardness in these inputs is highly localized in gadgets, with only a smallamount of complexity coming from global interactions, meaning that the relevance to real-worldproblems is limited.
In this study we provide a new synthetic problem class that addresses the limitations of the Googleinputs while retaining their strengths. We use simple clusters instead of more complex gadgets andmore emphasis is placed on creating computational hardness through frustrated global interactionslike those seen in interesting real-world inputs. The logical spin-glass backbones used to generatethese inputs are planar Ising models without fields and the problems can therefore be solved inpolynomial time [J. Phys. A, 15:10 (1982)]. However, for general heuristic algorithms that areunaware of the planted problem class, the frustration creates meaningful difficulty in a controlledenvironment ideal for study.
We use these inputs to evaluate the new 2000-qubit D-Wave QPU. We include the HFSalgorithm—the best performer in a broader analysis of Google inputs—and we include state-of-the-art GPU implementations of SA and QMC. The D-Wave QPU solidly outperforms the softwaresolvers: when we consider pure annealing time (computation time), the D-Wave QPU reaches groundstates up to 2600 times faster than the competition. In the task of zero-temperature Boltzmannsampling from challenging multimodal inputs, the D-Wave QPU holds a similar advantage and doesnot see significant performance degradation due to quantum sampling bias.
CONTENTS
I. Introduction 2
A. Proposing a new problem class 2
B. Evaluation of the 2000-qubit D-WaveQPU 2
II. D-Wave quantum processing units 3
A. Ising minimization 3
B. Chimera topology 3
C. Quantum annealing 3
D. Modeling performance 4
III. Frustrated Cluster Loop problems 5
A. Ruggedness and clusters 5
B. FCL problem generation 5
C. Problem class parameters 6D. Confirming correlation between
ruggedness and classical hardness 6
IV. Software solvers 7
V. Optimization 7
A. Varying ruggedness via logicalcomplexity 7
B. Varying ruggedness by scaling 8
VI. Sampling 9
A. Sampling from all valleys 9
B. Mining for interesting valley structure 9
C. Sampling results 10
VII. Conclusions 12
References 13
A. Calculation of decorrelation 14
B. Details of software solvers 15
1. Classical hardware 15
arX
iv:1
701.
0457
9v2
[qua
nt-p
h] 1
Mar
201
7
D-Wave Centric Approach to Benchmarking
Problem Generation
Test Case
HFS
Simulated Annealing
Benchmarking a quantum annealing processor with the
time-to-target metric.
James King⇤, Sheir Yarkoni, Mayssam M. Nevisi, Jeremy P. Hilton, andCatherine C. McGeoch
D-Wave Systems, Burnaby, BC
August 21, 2015
Abstract
In the evaluation of quantum annealers, metrics based on ground state success rates havetwo major drawbacks. First, evaluation requires computation time for both quantum andclassical processors that grows exponentially with problem size. This makes evaluation itselfcomputationally prohibitive. Second, results are heavily dependent on the e↵ects of analognoise on the quantum processors, which is an engineering issue that complicates the study ofthe underlying quantum annealing algorithm. We introduce a novel “time-to-target” metricwhich avoids these two issues by challenging software solvers to match the results obtained by aquantum annealer in a short amount of time. We evaluate D-Wave’s latest quantum annealer,the D-Wave 2X system, on an array of problem classes and find that it performs well on severalinput classes relative to state of the art software solvers running single-threaded on a CPU.
1 Introduction
The commercial availability of D-Wave’s quantum annealers1 in recent years [1, 2, 3] has led tomany interesting challenges in benchmarking, including the identification of suitable metrics forcomparing performance against classical solution methods. Several types of di�culties arise. First,a D-Wave computation is both quantum and analog, with nothing resembling a discrete instructionor basic operation that can be counted, as in classical benchmarking scenarios; with no commondenominator for comparison we resort to runtime measurements, which are notoriously transientand unstable. A second set of issues arises from the fact that we compare an algorithm implementedin hardware to algorithms implemented in software; standard guidelines for benchmarking computerplatforms, software, and algorithms ([4, 5, 6, 7]) do not consider this mixed scenario.
Another di�culty, addressed in this paper, arises from the fact that the algorithms of interestare heuristics for an NP-hard optimization problem. Unlike decision problems, where the algorithmreturns a solution that can be verified right or wrong, optimization problems allow heuristic solu-tions that may be better or worse, and that cannot be e�ciently verified as optimal (unless P=NP).
⇤Corresponding author, [email protected], D-Wave One, D-Wave Two, and D-Wave 2X are trademarks of D-Wave Systems Inc.
1
arX
iv:1
508.
0508
7v1
[qua
nt-p
h] 2
0 A
ug 2
015
What is the Computational Value of Finite Range Tunneling?
Vasil S. Denchev,1 Sergio Boixo,1 Sergei V. Isakov,1 Nan Ding,1 RyanBabbush,1 Vadim Smelyanskiy,1 John Martinis,2 and Hartmut Neven1
1Google Inc., Venice, CA 90291, USA2Google Inc., Santa Barbara, CA 93117, USA
(Dated: January 26, 2016)
Quantum annealing (QA) has been proposed as a quantum enhanced optimization heuristic ex-ploiting tunneling. Here, we demonstrate how finite range tunneling can provide considerable com-putational advantage. For a crafted problem designed to have tall and narrow energy barriersseparating local minima, the D-Wave 2X quantum annealer achieves significant runtime advantagesrelative to Simulated Annealing (SA). For instances with 945 variables, this results in a time-to-99%-success-probability that is ⇠ 108 times faster than SA running on a single processor core. We alsocompared physical QA with Quantum Monte Carlo (QMC), an algorithm that emulates quantumtunneling on classical processors. We observe a substantial constant overhead against physical QA:D-Wave 2X again runs up to ⇠ 108 times faster than an optimized implementation of QMC on asingle core. We note that there exist heuristic classical algorithms that can solve most instancesof Chimera structured problems in a timescale comparable to the D-Wave 2X. However, we believethat such solvers will become ine↵ective for the next generation of annealers currently being de-signed. To investigate whether finite range tunneling will also confer an advantage for problems ofpractical interest, we conduct numerical studies on binary optimization problems that cannot yetbe represented on quantum hardware. For random instances of the number partitioning problem,we find numerically that QMC, as well as other algorithms designed to simulate QA, scale betterthan SA. We discuss the implications of these findings for the design of next generation quantumannealers.
I. INTRODUCTION
Simulated annealing (SA) [1] is perhaps the mostwidely used algorithm for global optimization of pseudo-Boolean functions with little known structure. The ob-jective function for this general class of problems is
Hcl
P
(s) = �KX
k=1
NXj1...jk=1
Jj1···jksj1 · · · s
j
k
, (1)
where N is the problem size, sj
= ±1 are spin variablesand the couplings J
j1...jk are real scalars. In the physicsliterature Hcl
P
(s) is known as the Hamiltonian of a K-spinmodel. SA is a Monte Carlo algorithm designed to mimicthe thermalization dynamics of a system in contact witha slowly cooling reservoir. When the temperature is high,SA induces thermal excitations which can allow the sys-tem to escape from local minima. As the temperaturedecreases, SA drives the system towards nearby low en-ergy spin configurations.
Almost two decades ago, quantum annealing (QA) [2]was proposed as a heuristic technique for quantum en-hanced optimization. Despite substantial academic andindustrial interest [3–34], a unified understanding of thephysics of quantum annealing and its potential as an op-timization algorithm remains elusive. The appeal of QArelative to SA is due to the observation that quantummechanics allows for an additional escape route from lo-cal minima. While SA must climb over energy barriers toescape false traps, QA can penetrate these barriers with-out any increase in energy. This e↵ect is a hallmark ofquantum mechanics, known as quantum tunneling. The
standard time-dependent Hamiltonian used for QA is
H(t) = �A(t)NXj=1
�x
j
+ B(t)HP
, (2)
where HP
is written as in Eq. (1) but with the spin vari-ables s
j
replaced with �z
j
Pauli matrices acting on qubitj, and the functions A(t) and B(t) define the anneal-ing schedule parameterized in terms of time t 2 [0, T
QA
](see Fig. 7). These annealing schedules can be defined inmany di↵erent ways as long as the functions are smooth,A(0) ⌧ B(0) and A(T
QA
) � B(TQA
). At the beginningof the annealing, the transverse field magnitude A(t) islarge, and the system dynamics are dominated by quan-tum fluctuations due to tunneling (as opposed to thethermal fluctuations used in SA).
The question of whether D-Wave processors realizecomputationally relevant quantum tunneling has beena subject of substantial debate. This debate has nowbeen settled in the a�rmative with a sequence of publica-tions [6, 8, 9, 12, 13, 18, 21] demonstrating that quantumresources are present and functional in the processors. Inparticular, Refs. [35, 36] studied the performance of theD-Wave device on problems where eight [37] qubit co-tunneling events were employed in a functional mannerto reach low-lying energy solutions.
In order to investigate the computational value of fi-nite range tunneling in QA, we study the scaling of theexponential dependence of annealing time with the sizeof the tunneling domain D,
TQA
= BQA
e↵D , (3)
arX
iv:1
512.
0220
6v4
[qua
nt-p
h] 2
2 Ja
n 20
16
Quantum Annealing amid Local Ruggedness and Global Frustration
James King,⇤ Sheir Yarkoni, Jack Raymond, Isil Ozfidan, Andrew D. King,Mayssam Mohammadi Nevisi, Jeremy P. Hilton, and Catherine C. McGeoch
D-Wave Systems
(Dated: March 2, 2017)
A recent Google study [Phys. Rev. X, 6:031015 (2016)] compared a D-Wave 2X quantum process-ing unit (QPU) to two classical Monte Carlo algorithms: simulated annealing (SA) and quantumMonte Carlo (QMC). The study showed the D-Wave 2X to be up to 100 million times faster thanthe classical algorithms. The Google inputs are designed to demonstrate the value of collectivemultiqubit tunneling, a resource that is available to D-Wave QPUs but not to simulated annealing.But the computational hardness in these inputs is highly localized in gadgets, with only a smallamount of complexity coming from global interactions, meaning that the relevance to real-worldproblems is limited.
In this study we provide a new synthetic problem class that addresses the limitations of the Googleinputs while retaining their strengths. We use simple clusters instead of more complex gadgets andmore emphasis is placed on creating computational hardness through frustrated global interactionslike those seen in interesting real-world inputs. The logical spin-glass backbones used to generatethese inputs are planar Ising models without fields and the problems can therefore be solved inpolynomial time [J. Phys. A, 15:10 (1982)]. However, for general heuristic algorithms that areunaware of the planted problem class, the frustration creates meaningful difficulty in a controlledenvironment ideal for study.
We use these inputs to evaluate the new 2000-qubit D-Wave QPU. We include the HFSalgorithm—the best performer in a broader analysis of Google inputs—and we include state-of-the-art GPU implementations of SA and QMC. The D-Wave QPU solidly outperforms the softwaresolvers: when we consider pure annealing time (computation time), the D-Wave QPU reaches groundstates up to 2600 times faster than the competition. In the task of zero-temperature Boltzmannsampling from challenging multimodal inputs, the D-Wave QPU holds a similar advantage and doesnot see significant performance degradation due to quantum sampling bias.
CONTENTS
I. Introduction 2
A. Proposing a new problem class 2
B. Evaluation of the 2000-qubit D-WaveQPU 2
II. D-Wave quantum processing units 3
A. Ising minimization 3
B. Chimera topology 3
C. Quantum annealing 3
D. Modeling performance 4
III. Frustrated Cluster Loop problems 5
A. Ruggedness and clusters 5
B. FCL problem generation 5
C. Problem class parameters 6D. Confirming correlation between
ruggedness and classical hardness 6
IV. Software solvers 7
V. Optimization 7
A. Varying ruggedness via logicalcomplexity 7
B. Varying ruggedness by scaling 8
VI. Sampling 9
A. Sampling from all valleys 9
B. Mining for interesting valley structure 9
C. Sampling results 10
VII. Conclusions 12
References 13
A. Calculation of decorrelation 14
B. Details of software solvers 15
1. Classical hardware 15
arX
iv:1
701.
0457
9v2
[qua
nt-p
h] 1
Mar
201
7
D-Wave Centric Approach to Benchmarking
Problem Generation
Test Case
HFS
Simulated Annealing
Benchmarking a quantum annealing processor with the
time-to-target metric.
James King⇤, Sheir Yarkoni, Mayssam M. Nevisi, Jeremy P. Hilton, andCatherine C. McGeoch
D-Wave Systems, Burnaby, BC
August 21, 2015
Abstract
In the evaluation of quantum annealers, metrics based on ground state success rates havetwo major drawbacks. First, evaluation requires computation time for both quantum andclassical processors that grows exponentially with problem size. This makes evaluation itselfcomputationally prohibitive. Second, results are heavily dependent on the e↵ects of analognoise on the quantum processors, which is an engineering issue that complicates the study ofthe underlying quantum annealing algorithm. We introduce a novel “time-to-target” metricwhich avoids these two issues by challenging software solvers to match the results obtained by aquantum annealer in a short amount of time. We evaluate D-Wave’s latest quantum annealer,the D-Wave 2X system, on an array of problem classes and find that it performs well on severalinput classes relative to state of the art software solvers running single-threaded on a CPU.
1 Introduction
The commercial availability of D-Wave’s quantum annealers1 in recent years [1, 2, 3] has led tomany interesting challenges in benchmarking, including the identification of suitable metrics forcomparing performance against classical solution methods. Several types of di�culties arise. First,a D-Wave computation is both quantum and analog, with nothing resembling a discrete instructionor basic operation that can be counted, as in classical benchmarking scenarios; with no commondenominator for comparison we resort to runtime measurements, which are notoriously transientand unstable. A second set of issues arises from the fact that we compare an algorithm implementedin hardware to algorithms implemented in software; standard guidelines for benchmarking computerplatforms, software, and algorithms ([4, 5, 6, 7]) do not consider this mixed scenario.
Another di�culty, addressed in this paper, arises from the fact that the algorithms of interestare heuristics for an NP-hard optimization problem. Unlike decision problems, where the algorithmreturns a solution that can be verified right or wrong, optimization problems allow heuristic solu-tions that may be better or worse, and that cannot be e�ciently verified as optimal (unless P=NP).
⇤Corresponding author, [email protected], D-Wave One, D-Wave Two, and D-Wave 2X are trademarks of D-Wave Systems Inc.
1
arX
iv:1
508.
0508
7v1
[qua
nt-p
h] 2
0 A
ug 2
015
What is the Computational Value of Finite Range Tunneling?
Vasil S. Denchev,1 Sergio Boixo,1 Sergei V. Isakov,1 Nan Ding,1 RyanBabbush,1 Vadim Smelyanskiy,1 John Martinis,2 and Hartmut Neven1
1Google Inc., Venice, CA 90291, USA2Google Inc., Santa Barbara, CA 93117, USA
(Dated: January 26, 2016)
Quantum annealing (QA) has been proposed as a quantum enhanced optimization heuristic ex-ploiting tunneling. Here, we demonstrate how finite range tunneling can provide considerable com-putational advantage. For a crafted problem designed to have tall and narrow energy barriersseparating local minima, the D-Wave 2X quantum annealer achieves significant runtime advantagesrelative to Simulated Annealing (SA). For instances with 945 variables, this results in a time-to-99%-success-probability that is ⇠ 108 times faster than SA running on a single processor core. We alsocompared physical QA with Quantum Monte Carlo (QMC), an algorithm that emulates quantumtunneling on classical processors. We observe a substantial constant overhead against physical QA:D-Wave 2X again runs up to ⇠ 108 times faster than an optimized implementation of QMC on asingle core. We note that there exist heuristic classical algorithms that can solve most instancesof Chimera structured problems in a timescale comparable to the D-Wave 2X. However, we believethat such solvers will become ine↵ective for the next generation of annealers currently being de-signed. To investigate whether finite range tunneling will also confer an advantage for problems ofpractical interest, we conduct numerical studies on binary optimization problems that cannot yetbe represented on quantum hardware. For random instances of the number partitioning problem,we find numerically that QMC, as well as other algorithms designed to simulate QA, scale betterthan SA. We discuss the implications of these findings for the design of next generation quantumannealers.
I. INTRODUCTION
Simulated annealing (SA) [1] is perhaps the mostwidely used algorithm for global optimization of pseudo-Boolean functions with little known structure. The ob-jective function for this general class of problems is
Hcl
P
(s) = �KX
k=1
NXj1...jk=1
Jj1···jksj1 · · · s
j
k
, (1)
where N is the problem size, sj
= ±1 are spin variablesand the couplings J
j1...jk are real scalars. In the physicsliterature Hcl
P
(s) is known as the Hamiltonian of a K-spinmodel. SA is a Monte Carlo algorithm designed to mimicthe thermalization dynamics of a system in contact witha slowly cooling reservoir. When the temperature is high,SA induces thermal excitations which can allow the sys-tem to escape from local minima. As the temperaturedecreases, SA drives the system towards nearby low en-ergy spin configurations.
Almost two decades ago, quantum annealing (QA) [2]was proposed as a heuristic technique for quantum en-hanced optimization. Despite substantial academic andindustrial interest [3–34], a unified understanding of thephysics of quantum annealing and its potential as an op-timization algorithm remains elusive. The appeal of QArelative to SA is due to the observation that quantummechanics allows for an additional escape route from lo-cal minima. While SA must climb over energy barriers toescape false traps, QA can penetrate these barriers with-out any increase in energy. This e↵ect is a hallmark ofquantum mechanics, known as quantum tunneling. The
standard time-dependent Hamiltonian used for QA is
H(t) = �A(t)NXj=1
�x
j
+ B(t)HP
, (2)
where HP
is written as in Eq. (1) but with the spin vari-ables s
j
replaced with �z
j
Pauli matrices acting on qubitj, and the functions A(t) and B(t) define the anneal-ing schedule parameterized in terms of time t 2 [0, T
QA
](see Fig. 7). These annealing schedules can be defined inmany di↵erent ways as long as the functions are smooth,A(0) ⌧ B(0) and A(T
QA
) � B(TQA
). At the beginningof the annealing, the transverse field magnitude A(t) islarge, and the system dynamics are dominated by quan-tum fluctuations due to tunneling (as opposed to thethermal fluctuations used in SA).
The question of whether D-Wave processors realizecomputationally relevant quantum tunneling has beena subject of substantial debate. This debate has nowbeen settled in the a�rmative with a sequence of publica-tions [6, 8, 9, 12, 13, 18, 21] demonstrating that quantumresources are present and functional in the processors. Inparticular, Refs. [35, 36] studied the performance of theD-Wave device on problems where eight [37] qubit co-tunneling events were employed in a functional mannerto reach low-lying energy solutions.
In order to investigate the computational value of fi-nite range tunneling in QA, we study the scaling of theexponential dependence of annealing time with the sizeof the tunneling domain D,
TQA
= BQA
e↵D , (3)
arX
iv:1
512.
0220
6v4
[qua
nt-p
h] 2
2 Ja
n 20
16
Quantum Annealing amid Local Ruggedness and Global Frustration
James King,⇤ Sheir Yarkoni, Jack Raymond, Isil Ozfidan, Andrew D. King,Mayssam Mohammadi Nevisi, Jeremy P. Hilton, and Catherine C. McGeoch
D-Wave Systems
(Dated: March 2, 2017)
A recent Google study [Phys. Rev. X, 6:031015 (2016)] compared a D-Wave 2X quantum process-ing unit (QPU) to two classical Monte Carlo algorithms: simulated annealing (SA) and quantumMonte Carlo (QMC). The study showed the D-Wave 2X to be up to 100 million times faster thanthe classical algorithms. The Google inputs are designed to demonstrate the value of collectivemultiqubit tunneling, a resource that is available to D-Wave QPUs but not to simulated annealing.But the computational hardness in these inputs is highly localized in gadgets, with only a smallamount of complexity coming from global interactions, meaning that the relevance to real-worldproblems is limited.
In this study we provide a new synthetic problem class that addresses the limitations of the Googleinputs while retaining their strengths. We use simple clusters instead of more complex gadgets andmore emphasis is placed on creating computational hardness through frustrated global interactionslike those seen in interesting real-world inputs. The logical spin-glass backbones used to generatethese inputs are planar Ising models without fields and the problems can therefore be solved inpolynomial time [J. Phys. A, 15:10 (1982)]. However, for general heuristic algorithms that areunaware of the planted problem class, the frustration creates meaningful difficulty in a controlledenvironment ideal for study.
We use these inputs to evaluate the new 2000-qubit D-Wave QPU. We include the HFSalgorithm—the best performer in a broader analysis of Google inputs—and we include state-of-the-art GPU implementations of SA and QMC. The D-Wave QPU solidly outperforms the softwaresolvers: when we consider pure annealing time (computation time), the D-Wave QPU reaches groundstates up to 2600 times faster than the competition. In the task of zero-temperature Boltzmannsampling from challenging multimodal inputs, the D-Wave QPU holds a similar advantage and doesnot see significant performance degradation due to quantum sampling bias.
CONTENTS
I. Introduction 2
A. Proposing a new problem class 2
B. Evaluation of the 2000-qubit D-WaveQPU 2
II. D-Wave quantum processing units 3
A. Ising minimization 3
B. Chimera topology 3
C. Quantum annealing 3
D. Modeling performance 4
III. Frustrated Cluster Loop problems 5
A. Ruggedness and clusters 5
B. FCL problem generation 5
C. Problem class parameters 6D. Confirming correlation between
ruggedness and classical hardness 6
IV. Software solvers 7
V. Optimization 7
A. Varying ruggedness via logicalcomplexity 7
B. Varying ruggedness by scaling 8
VI. Sampling 9
A. Sampling from all valleys 9
B. Mining for interesting valley structure 9
C. Sampling results 10
VII. Conclusions 12
References 13
A. Calculation of decorrelation 14
B. Details of software solvers 15
1. Classical hardware 15
arX
iv:1
701.
0457
9v2
[qua
nt-p
h] 1
Mar
201
7
D-Wave Centric Approach to Benchmarking
or-tools
Problem Generation
Test Case
If this is the standard benchmarking procedure, it’s worthwhile to build some reusable tools
D-Wave Centric Approach to Benchmarking
or-tools
Problem Generation
Test Case
BQPSolvers
DWIG
BQPJSON
If this is the standard benchmarking procedure, it’s worthwhile to build some reusable tools
D-Wave Centric Approach to Benchmarking
or-tools
Problem Generation
Test Case
BQPSolvers
DWIG
BQPJSON Start Here
If this is the standard benchmarking procedure, it’s worthwhile to build some reusable tools
B-QP Test Case Key Features
Baseline Requirement: If you have access to my QPU, you can easily replicate my results
B-QP Test Case Key Features
• Variable space agnostic • spin {-1,1} or boolean {0,1} Baseline Requirement:
If you have access to my QPU, you can easily replicate my results
B-QP Test Case Key Features
• Variable space agnostic • spin {-1,1} or boolean {0,1}
• Variable names are important • no embedding required • not all qubits are created equal
Baseline Requirement: If you have access to my QPU, you can easily replicate my results
B-QP Test Case Key Features
• Variable space agnostic • spin {-1,1} or boolean {0,1}
• Variable names are important • no embedding required • not all qubits are created equal
• Rescaling is important • problem units vs machine units • unit scale is essential for sampling
Baseline Requirement: If you have access to my QPU, you can easily replicate my results
B-QP Test Case Key Features
• Variable space agnostic • spin {-1,1} or boolean {0,1}
• Variable names are important • no embedding required • not all qubits are created equal
• Rescaling is important • problem units vs machine units • unit scale is essential for sampling
• Metadata is helpful • solver url • solver name • qpu chip id
Baseline Requirement: If you have access to my QPU, you can easily replicate my results
B-QP Test Case Key Features
• Variable space agnostic • spin {-1,1} or boolean {0,1}
• Variable names are important • no embedding required • not all qubits are created equal
• Rescaling is important • problem units vs machine units • unit scale is essential for sampling
• Metadata is helpful • solver url • solver name • qpu chip id
Baseline Requirement: If you have access to my QPU, you can easily replicate my results
Unable to find a data format that supported all of these features…
B-QP Test Case Key Features
• Variable space agnostic • spin {-1,1} or boolean {0,1}
• Variable names are important • no embedding required • not all qubits are created equal
• Rescaling is important • problem units vs machine units • unit scale is essential for sampling
• Metadata is helpful • solver url • solver name • qpu chip id
BQPJSONA JSON-base data
format for unconstrained binary quadratic programs
Baseline Requirement: If you have access to my QPU, you can easily replicate my results
Unable to find a data format that supported all of these features…
BQPJSON - Mathematical Model
min : s
0
@X
i,j2Ecijbibj +
X
i2Ncibi + o
1
A
s.t.: bi 2 {�1, 1} 8i 2 N
min : s
0
@X
i,j2Ecijbibj +
X
i2Ncibi + o
1
A
s.t.: bi 2 {0, 1} 8i 2 N
Ising Formulation
QUBO Formulation
One Data Format Two Mathematical Models
BQPJSON - Mathematical Model
min : s
0
@X
i,j2Ecijbibj +
X
i2Ncibi + o
1
A
s.t.: bi 2 {�1, 1} 8i 2 N
min : s
0
@X
i,j2Ecijbibj +
X
i2Ncibi + o
1
A
s.t.: bi 2 {0, 1} 8i 2 N
J
Ising Formulation
QUBO Formulation
One Data Format Two Mathematical Models
BQPJSON - Mathematical Model
min : s
0
@X
i,j2Ecijbibj +
X
i2Ncibi + o
1
A
s.t.: bi 2 {�1, 1} 8i 2 N
min : s
0
@X
i,j2Ecijbibj +
X
i2Ncibi + o
1
A
s.t.: bi 2 {0, 1} 8i 2 N
hJ
Ising Formulation
QUBO Formulation
One Data Format Two Mathematical Models
BQPJSON - Mathematical Model
min : s
0
@X
i,j2Ecijbibj +
X
i2Ncibi + o
1
A
s.t.: bi 2 {�1, 1} 8i 2 N
min : s
0
@X
i,j2Ecijbibj +
X
i2Ncibi + o
1
A
s.t.: bi 2 {0, 1} 8i 2 N
offsethJ
Ising Formulation
QUBO Formulation
One Data Format Two Mathematical Models
BQPJSON - Mathematical Model
min : s
0
@X
i,j2Ecijbibj +
X
i2Ncibi + o
1
A
s.t.: bi 2 {�1, 1} 8i 2 N
min : s
0
@X
i,j2Ecijbibj +
X
i2Ncibi + o
1
A
s.t.: bi 2 {0, 1} 8i 2 N
offsethJ
InvertibleIsing Formulation
QUBO Formulation
One Data Format Two Mathematical Models
BQPJSON - Mathematical Model
min : s
0
@X
i,j2Ecijbibj +
X
i2Ncibi + o
1
A
s.t.: bi 2 {�1, 1} 8i 2 N
min : s
0
@X
i,j2Ecijbibj +
X
i2Ncibi + o
1
A
s.t.: bi 2 {0, 1} 8i 2 N
offsethJscalar
InvertibleIsing Formulation
QUBO Formulation
One Data Format Two Mathematical Models
BQPJSON - Mathematical Model
min : s
0
@X
i,j2Ecijbibj +
X
i2Ncibi + o
1
A
s.t.: bi 2 {�1, 1} 8i 2 N
min : s
0
@X
i,j2Ecijbibj +
X
i2Ncibi + o
1
A
s.t.: bi 2 {0, 1} 8i 2 N
offsethJscalar
InvertibleIsing Formulation
QUBO Formulation
One Data Format Two Mathematical Models
hardware units
BQPJSON - Mathematical Model
min : s
0
@X
i,j2Ecijbibj +
X
i2Ncibi + o
1
A
s.t.: bi 2 {�1, 1} 8i 2 N
min : s
0
@X
i,j2Ecijbibj +
X
i2Ncibi + o
1
A
s.t.: bi 2 {0, 1} 8i 2 N
offsethJscalar
InvertibleIsing Formulation
QUBO Formulation
One Data Format Two Mathematical Models
min : �0.2b2b4 + 1.5b2b6 + 1.3b2 � 0.7b6
s.t.: bi 2 {�1, 1} 8i 2 {2, 4, 6}
Examplehardware
units
Why JSON?
• JSON = Java Script Object Notation • very similar to python dictionaries
{ "key_1": true, "key_2": [1, 2.5, "some string"], "key_3": { "nested_key”:"nested_value" }}
Why JSON?
• JSON = Java Script Object Notation • very similar to python dictionaries
• Super simple design
{ "key_1": true, "key_2": [1, 2.5, "some string"], "key_3": { "nested_key”:"nested_value" }}
Why JSON?
• JSON = Java Script Object Notation • very similar to python dictionaries
• Super simple design• Allows for hierarchical data
organization • beyond csv table-like data
{ "key_1": true, "key_2": [1, 2.5, "some string"], "key_3": { "nested_key”:"nested_value" }}
Why JSON?
• JSON = Java Script Object Notation • very similar to python dictionaries
• Super simple design• Allows for hierarchical data
organization • beyond csv table-like data
• Every programming language has a great JSON parser
{ "key_1": true, "key_2": [1, 2.5, "some string"], "key_3": { "nested_key”:"nested_value" }}
BQPJSON - Example
{ "description":"a simple model", "id": 0, "linear_terms": [ {"coeff": 1.3, "id": 2}, {"coeff": -0.7, "id": 6} ], "metadata": {}, "offset": 0.0, "quadratic_terms": [ {"coeff": -0.2, "id_head": 4, "id_tail": 2}, {"coeff": 1.5, "id_head": 6, "id_tail": 2} ], "scale": 1.0, "variable_domain": "spin", "variable_ids": [2,4,6], "version": "1.0.0"}
min : �0.2b2b4 + 1.5b2b6 + 1.3b2 � 0.7b6
s.t.: bi 2 {�1, 1} 8i 2 {2, 4, 6}
BQPJSON - Example
{ "description":"a simple model", "id": 0, "linear_terms": [ {"coeff": 1.3, "id": 2}, {"coeff": -0.7, "id": 6} ], "metadata": {}, "offset": 0.0, "quadratic_terms": [ {"coeff": -0.2, "id_head": 4, "id_tail": 2}, {"coeff": 1.5, "id_head": 6, "id_tail": 2} ], "scale": 1.0, "variable_domain": "spin", "variable_ids": [2,4,6], "version": "1.0.0"}
min : �0.2b2b4 + 1.5b2b6 + 1.3b2 � 0.7b6
s.t.: bi 2 {�1, 1} 8i 2 {2, 4, 6}
BQPJSON - Example
{ "description":"a simple model", "id": 0, "linear_terms": [ {"coeff": 1.3, "id": 2}, {"coeff": -0.7, "id": 6} ], "metadata": {}, "offset": 0.0, "quadratic_terms": [ {"coeff": -0.2, "id_head": 4, "id_tail": 2}, {"coeff": 1.5, "id_head": 6, "id_tail": 2} ], "scale": 1.0, "variable_domain": "spin", "variable_ids": [2,4,6], "version": "1.0.0"}
min : �0.2b2b4 + 1.5b2b6 + 1.3b2 � 0.7b6
s.t.: bi 2 {�1, 1} 8i 2 {2, 4, 6}
BQPJSON - Example
{ "description":"a simple model", "id": 0, "linear_terms": [ {"coeff": 1.3, "id": 2}, {"coeff": -0.7, "id": 6} ], "metadata": {}, "offset": 0.0, "quadratic_terms": [ {"coeff": -0.2, "id_head": 4, "id_tail": 2}, {"coeff": 1.5, "id_head": 6, "id_tail": 2} ], "scale": 1.0, "variable_domain": "spin", "variable_ids": [2,4,6], "version": "1.0.0"}
min : �0.2b2b4 + 1.5b2b6 + 1.3b2 � 0.7b6
s.t.: bi 2 {�1, 1} 8i 2 {2, 4, 6}
BQPJSON - Example
{ "description":"a simple model", "id": 0, "linear_terms": [ {"coeff": 1.3, "id": 2}, {"coeff": -0.7, "id": 6} ], "metadata": {}, "offset": 0.0, "quadratic_terms": [ {"coeff": -0.2, "id_head": 4, "id_tail": 2}, {"coeff": 1.5, "id_head": 6, "id_tail": 2} ], "scale": 1.0, "variable_domain": "spin", "variable_ids": [2,4,6], "version": "1.0.0"}
min : �0.2b2b4 + 1.5b2b6 + 1.3b2 � 0.7b6
s.t.: bi 2 {�1, 1} 8i 2 {2, 4, 6}
BQPJSON - Other Useful Features
• Advanced Metadata Features • chimera graph structure annotation
• no need to reverse engineer the graph structure (e.g. HFS) • solver parameters (e.g. annealing time, spin reversal transform)
BQPJSON - Other Useful Features
• Advanced Metadata Features • chimera graph structure annotation
• no need to reverse engineer the graph structure (e.g. HFS) • solver parameters (e.g. annealing time, spin reversal transform)
• Solution Encoding • easily share best-known variable assignments • very helpful when the test case has a planted ground state
BQPJSON - More than a Data Format
• Python package with useful tools
pip install bqpjson
https://github.com/lanl-ansi/bqpjson
BQPJSON - More than a Data Format
• Python package with useful tools• Data validation
• check if JSON data is BQPJSON data
pip install bqpjson
https://github.com/lanl-ansi/bqpjson
BQPJSON - More than a Data Format
• Python package with useful tools• Data validation
• check if JSON data is BQPJSON data• Command line tools
• spin2bool - swap variables • bqp2qh - past into qubist • bqp2qubo - qbsolv format • bqp2hfs - HFS format • bqp2mzn - MiniZinc format
pip install bqpjson
https://github.com/lanl-ansi/bqpjson
BQPJSON - More than a Data Format
• Python package with useful tools• Data validation
• check if JSON data is BQPJSON data• Command line tools
• spin2bool - swap variables • bqp2qh - past into qubist • bqp2qubo - qbsolv format • bqp2hfs - HFS format • bqp2mzn - MiniZinc format
pip install bqpjson
https://github.com/lanl-ansi/bqpjson
cat ising1.json | spin2bool > qubo1.json
cat qubo1.json | spin2bool > ising2.json
BQPJSON
BQPJSON - More than a Data Format
• Python package with useful tools• Data validation
• check if JSON data is BQPJSON data• Command line tools
• spin2bool - swap variables • bqp2qh - past into qubist • bqp2qubo - qbsolv format • bqp2hfs - HFS format • bqp2mzn - MiniZinc format
pip install bqpjson
https://github.com/lanl-ansi/bqpjson
cat ising1.json | spin2bool > qubo1.json
cat qubo1.json | spin2bool > ising2.json
BQPJSON
Identical
D-Wave Centric Approach to Benchmarking
or-tools
Problem Generation
BQP Test Case
BQPSolvers
DWIG
BQPJSON
And Now This
D-Wave Instance Generator (DWIG)
• At QPU in, Test case out
D-Wave Instance
Generator (DWIG)
D-Wave Instance Generator (DWIG)
• At QPU in, Test case out• But problem generation is a tricky business!
D-Wave Instance
Generator (DWIG)
as ist
avid itchell Dept. of Computing Science AT&T Bell Laboratories
Simon Fraser University Murray Hill, NJ 07974
Burnaby, Canada V5A lS6 selmanQresearch.att.com
mitchellQcs.sfu.ca
Abstract
We report results from large-scale experiments in satisfiability testing. As has been observed by others, testing the satisfiability of random formu- las often appears surprisingly easy. Here we show that by using the right distribution of instances, and appropriate parameter values, it is possible to generate random formulas that are hard, that is, for which satisfiability testing is quite difficult. Our results provide a benchmark for the evalua- tion of satisfiability-testing procedures.
Introduction Many computational tasks of interest to AI, to the ex- tent that they can be precisely characterized at all, can be shown to be NP-hard in their most general form. However, there is fundamental disagreement, at
least within the AI community, about the implications of this. It is claimed on the one hand that since the performance of algorithms designed to solve NP-hard tasks degrades rapidly with small increases in input size, something will need to be given up to obtain ac- ceptable behavior. On the other hand, it is argued that this analysis is irrelevant to AI since it based on worst-case scenarios, and that what is really needed is a better understanding of how these procedures per- form “on average”.
The first computational task shown to be NP-hard, by Cook (1971) was propositional satisfiability or SAT: given a formula of the propositional calculus, de- cide if there is an assignment to its variables that makes the formula true according to the usual rules of inter- pretation. Subsequent tasks have been shown to be NP-hard by proving they are at least as hard as SAT. Roughly, a task is NP-hard if a good algorithm for it would entail a good algorithm for SAT. Unlike many other NP-hard tasks (see Garey and Johnson (1979) for a catalogue), SAT is of special concern to AI because of its direct relationship to deductive reasoning (i.e.,
*Fellow of the Canadian Institute for Advanced Re- search, and E. W. R. Steacie Fellow of the Natural Sciences and Engineering Research Council of Canada
Dept. of Computer Science
University of Toron to
Toronto, Canada M5S lA4
hector8ai. toronto.edn
given a collection of base facts C, a sentence cy may be deduced iff C U {lo} is not satisfiable). Many other forms of reasoning, including default reasoning, diag- nosis, planning and image interpretation, also make
direct appeal to satisfiability. The fact that these usu- ally require much more than the propositional calculus simply highlights the fact that SAT is a fundamental task, and that developing SAT procedures that work well in AI applications is essential.
We might ask when it is reasonable to use a sound and complete procedure for SAT, and when we should settle for something less. Do hard cases come up often, or are they always a result of strange encodings tailored for some specific purpose ? One difficulty in answering such questions is that there appear to be few applica ble analytical results on the expected difficulty of SAT (although see below). It seems that, at least for the time being, we must rely largely on empirical results.
A number of papers (some discussed below) have claimed that the difficulty of SAT on randomly gen- erated problems is not so daunting. For example, an often-quoted result (Goldberg, 1979; Goldberg et al. 1982) suggests that SAT can be readily solved “on av- erage” in 0(n2) time. This does not settle the question of how well the methods will work in practice, but at first blush it does appear to be more relevant to AI than contrived worst cases.
The big problem is that to examine how well a pro- cedure does on average one must assume a distribution of instances. Indeed, as we will discuss below, Franc0 and Paul1 (1983) refuted the Goldberg result by show- ing that it was a direct consequence of their choice of distribution. It’s not that Goldberg had a clever al- gorithm, or that the problem is easy, but that they had used a distribution with a preponderance of easy instances. That is, from the space of all problem in- stances, they sampled in a way that produced almost no hard cases.
Nevertheless, papers continue to appear purport- ing to empirically demonstrate the efficacy of some new procedure, but using just this distribution (e.g., Hooker, 1988; Kamath et al. 1990), or presenting data suggesting that very large satisfiability problems -
Mitchell, Selman, and Levesque 459
From: AAAI-92 Proceedings. Copyright ©1992, AAAI (www.aaai.org). All rights reserved.
AAAI-92
D-Wave Instance Generator (DWIG)
• At QPU in, Test case out• But problem generation is a tricky business!
D-Wave Instance
Generator (DWIG)
as ist
avid itchell Dept. of Computing Science AT&T Bell Laboratories
Simon Fraser University Murray Hill, NJ 07974
Burnaby, Canada V5A lS6 selmanQresearch.att.com
mitchellQcs.sfu.ca
Abstract
We report results from large-scale experiments in satisfiability testing. As has been observed by others, testing the satisfiability of random formu- las often appears surprisingly easy. Here we show that by using the right distribution of instances, and appropriate parameter values, it is possible to generate random formulas that are hard, that is, for which satisfiability testing is quite difficult. Our results provide a benchmark for the evalua- tion of satisfiability-testing procedures.
Introduction Many computational tasks of interest to AI, to the ex- tent that they can be precisely characterized at all, can be shown to be NP-hard in their most general form. However, there is fundamental disagreement, at
least within the AI community, about the implications of this. It is claimed on the one hand that since the performance of algorithms designed to solve NP-hard tasks degrades rapidly with small increases in input size, something will need to be given up to obtain ac- ceptable behavior. On the other hand, it is argued that this analysis is irrelevant to AI since it based on worst-case scenarios, and that what is really needed is a better understanding of how these procedures per- form “on average”.
The first computational task shown to be NP-hard, by Cook (1971) was propositional satisfiability or SAT: given a formula of the propositional calculus, de- cide if there is an assignment to its variables that makes the formula true according to the usual rules of inter- pretation. Subsequent tasks have been shown to be NP-hard by proving they are at least as hard as SAT. Roughly, a task is NP-hard if a good algorithm for it would entail a good algorithm for SAT. Unlike many other NP-hard tasks (see Garey and Johnson (1979) for a catalogue), SAT is of special concern to AI because of its direct relationship to deductive reasoning (i.e.,
*Fellow of the Canadian Institute for Advanced Re- search, and E. W. R. Steacie Fellow of the Natural Sciences and Engineering Research Council of Canada
Dept. of Computer Science
University of Toron to
Toronto, Canada M5S lA4
hector8ai. toronto.edn
given a collection of base facts C, a sentence cy may be deduced iff C U {lo} is not satisfiable). Many other forms of reasoning, including default reasoning, diag- nosis, planning and image interpretation, also make
direct appeal to satisfiability. The fact that these usu- ally require much more than the propositional calculus simply highlights the fact that SAT is a fundamental task, and that developing SAT procedures that work well in AI applications is essential.
We might ask when it is reasonable to use a sound and complete procedure for SAT, and when we should settle for something less. Do hard cases come up often, or are they always a result of strange encodings tailored for some specific purpose ? One difficulty in answering such questions is that there appear to be few applica ble analytical results on the expected difficulty of SAT (although see below). It seems that, at least for the time being, we must rely largely on empirical results.
A number of papers (some discussed below) have claimed that the difficulty of SAT on randomly gen- erated problems is not so daunting. For example, an often-quoted result (Goldberg, 1979; Goldberg et al. 1982) suggests that SAT can be readily solved “on av- erage” in 0(n2) time. This does not settle the question of how well the methods will work in practice, but at first blush it does appear to be more relevant to AI than contrived worst cases.
The big problem is that to examine how well a pro- cedure does on average one must assume a distribution of instances. Indeed, as we will discuss below, Franc0 and Paul1 (1983) refuted the Goldberg result by show- ing that it was a direct consequence of their choice of distribution. It’s not that Goldberg had a clever al- gorithm, or that the problem is easy, but that they had used a distribution with a preponderance of easy instances. That is, from the space of all problem in- stances, they sampled in a way that produced almost no hard cases.
Nevertheless, papers continue to appear purport- ing to empirically demonstrate the efficacy of some new procedure, but using just this distribution (e.g., Hooker, 1988; Kamath et al. 1990), or presenting data suggesting that very large satisfiability problems -
Mitchell, Selman, and Levesque 459
From: AAAI-92 Proceedings. Copyright ©1992, AAAI (www.aaai.org). All rights reserved.
AAAI-92
Benchmarking a quantum annealing processor with the
time-to-target metric.
James King⇤, Sheir Yarkoni, Mayssam M. Nevisi, Jeremy P. Hilton, andCatherine C. McGeoch
D-Wave Systems, Burnaby, BC
August 21, 2015
Abstract
In the evaluation of quantum annealers, metrics based on ground state success rates havetwo major drawbacks. First, evaluation requires computation time for both quantum andclassical processors that grows exponentially with problem size. This makes evaluation itselfcomputationally prohibitive. Second, results are heavily dependent on the e↵ects of analognoise on the quantum processors, which is an engineering issue that complicates the study ofthe underlying quantum annealing algorithm. We introduce a novel “time-to-target” metricwhich avoids these two issues by challenging software solvers to match the results obtained by aquantum annealer in a short amount of time. We evaluate D-Wave’s latest quantum annealer,the D-Wave 2X system, on an array of problem classes and find that it performs well on severalinput classes relative to state of the art software solvers running single-threaded on a CPU.
1 Introduction
The commercial availability of D-Wave’s quantum annealers1 in recent years [1, 2, 3] has led tomany interesting challenges in benchmarking, including the identification of suitable metrics forcomparing performance against classical solution methods. Several types of di�culties arise. First,a D-Wave computation is both quantum and analog, with nothing resembling a discrete instructionor basic operation that can be counted, as in classical benchmarking scenarios; with no commondenominator for comparison we resort to runtime measurements, which are notoriously transientand unstable. A second set of issues arises from the fact that we compare an algorithm implementedin hardware to algorithms implemented in software; standard guidelines for benchmarking computerplatforms, software, and algorithms ([4, 5, 6, 7]) do not consider this mixed scenario.
Another di�culty, addressed in this paper, arises from the fact that the algorithms of interestare heuristics for an NP-hard optimization problem. Unlike decision problems, where the algorithmreturns a solution that can be verified right or wrong, optimization problems allow heuristic solu-tions that may be better or worse, and that cannot be e�ciently verified as optimal (unless P=NP).
⇤Corresponding author, [email protected], D-Wave One, D-Wave Two, and D-Wave 2X are trademarks of D-Wave Systems Inc.
1
arX
iv:1
508.
0508
7v1
[qua
nt-p
h] 2
0 A
ug 2
015
RAN, Frustrated Loops
Quantum Annealing amid Local Ruggedness and Global Frustration
James King,⇤ Sheir Yarkoni, Jack Raymond, Isil Ozfidan, Andrew D. King,Mayssam Mohammadi Nevisi, Jeremy P. Hilton, and Catherine C. McGeoch
D-Wave Systems
(Dated: March 2, 2017)
A recent Google study [Phys. Rev. X, 6:031015 (2016)] compared a D-Wave 2X quantum process-ing unit (QPU) to two classical Monte Carlo algorithms: simulated annealing (SA) and quantumMonte Carlo (QMC). The study showed the D-Wave 2X to be up to 100 million times faster thanthe classical algorithms. The Google inputs are designed to demonstrate the value of collectivemultiqubit tunneling, a resource that is available to D-Wave QPUs but not to simulated annealing.But the computational hardness in these inputs is highly localized in gadgets, with only a smallamount of complexity coming from global interactions, meaning that the relevance to real-worldproblems is limited.
In this study we provide a new synthetic problem class that addresses the limitations of the Googleinputs while retaining their strengths. We use simple clusters instead of more complex gadgets andmore emphasis is placed on creating computational hardness through frustrated global interactionslike those seen in interesting real-world inputs. The logical spin-glass backbones used to generatethese inputs are planar Ising models without fields and the problems can therefore be solved inpolynomial time [J. Phys. A, 15:10 (1982)]. However, for general heuristic algorithms that areunaware of the planted problem class, the frustration creates meaningful difficulty in a controlledenvironment ideal for study.
We use these inputs to evaluate the new 2000-qubit D-Wave QPU. We include the HFSalgorithm—the best performer in a broader analysis of Google inputs—and we include state-of-the-art GPU implementations of SA and QMC. The D-Wave QPU solidly outperforms the softwaresolvers: when we consider pure annealing time (computation time), the D-Wave QPU reaches groundstates up to 2600 times faster than the competition. In the task of zero-temperature Boltzmannsampling from challenging multimodal inputs, the D-Wave QPU holds a similar advantage and doesnot see significant performance degradation due to quantum sampling bias.
CONTENTS
I. Introduction 2
A. Proposing a new problem class 2B. Evaluation of the 2000-qubit D-Wave
QPU 2
II. D-Wave quantum processing units 3
A. Ising minimization 3B. Chimera topology 3
C. Quantum annealing 3D. Modeling performance 4
III. Frustrated Cluster Loop problems 5
A. Ruggedness and clusters 5B. FCL problem generation 5
C. Problem class parameters 6D. Confirming correlation between
ruggedness and classical hardness 6
IV. Software solvers 7
V. Optimization 7
A. Varying ruggedness via logicalcomplexity 7
B. Varying ruggedness by scaling 8
VI. Sampling 9A. Sampling from all valleys 9
B. Mining for interesting valley structure 9C. Sampling results 10
VII. Conclusions 12
References 13
A. Calculation of decorrelation 14
B. Details of software solvers 151. Classical hardware 15
arX
iv:1
701.
0457
9v2
[qua
nt-p
h] 1
Mar
201
7
Frustrated Cluster Loops
What is the Computational Value of Finite Range Tunneling?
Vasil S. Denchev,1 Sergio Boixo,1 Sergei V. Isakov,1 Nan Ding,1 RyanBabbush,1 Vadim Smelyanskiy,1 John Martinis,2 and Hartmut Neven1
1Google Inc., Venice, CA 90291, USA2Google Inc., Santa Barbara, CA 93117, USA
(Dated: January 26, 2016)
Quantum annealing (QA) has been proposed as a quantum enhanced optimization heuristic ex-ploiting tunneling. Here, we demonstrate how finite range tunneling can provide considerable com-putational advantage. For a crafted problem designed to have tall and narrow energy barriersseparating local minima, the D-Wave 2X quantum annealer achieves significant runtime advantagesrelative to Simulated Annealing (SA). For instances with 945 variables, this results in a time-to-99%-success-probability that is ⇠ 108 times faster than SA running on a single processor core. We alsocompared physical QA with Quantum Monte Carlo (QMC), an algorithm that emulates quantumtunneling on classical processors. We observe a substantial constant overhead against physical QA:D-Wave 2X again runs up to ⇠ 108 times faster than an optimized implementation of QMC on asingle core. We note that there exist heuristic classical algorithms that can solve most instancesof Chimera structured problems in a timescale comparable to the D-Wave 2X. However, we believethat such solvers will become ine↵ective for the next generation of annealers currently being de-signed. To investigate whether finite range tunneling will also confer an advantage for problems ofpractical interest, we conduct numerical studies on binary optimization problems that cannot yetbe represented on quantum hardware. For random instances of the number partitioning problem,we find numerically that QMC, as well as other algorithms designed to simulate QA, scale betterthan SA. We discuss the implications of these findings for the design of next generation quantumannealers.
I. INTRODUCTION
Simulated annealing (SA) [1] is perhaps the mostwidely used algorithm for global optimization of pseudo-Boolean functions with little known structure. The ob-jective function for this general class of problems is
Hcl
P
(s) = �KX
k=1
NXj1...jk=1
Jj1···jksj1 · · · s
j
k
, (1)
where N is the problem size, sj
= ±1 are spin variablesand the couplings J
j1...jk are real scalars. In the physicsliterature Hcl
P
(s) is known as the Hamiltonian of a K-spinmodel. SA is a Monte Carlo algorithm designed to mimicthe thermalization dynamics of a system in contact witha slowly cooling reservoir. When the temperature is high,SA induces thermal excitations which can allow the sys-tem to escape from local minima. As the temperaturedecreases, SA drives the system towards nearby low en-ergy spin configurations.
Almost two decades ago, quantum annealing (QA) [2]was proposed as a heuristic technique for quantum en-hanced optimization. Despite substantial academic andindustrial interest [3–34], a unified understanding of thephysics of quantum annealing and its potential as an op-timization algorithm remains elusive. The appeal of QArelative to SA is due to the observation that quantummechanics allows for an additional escape route from lo-cal minima. While SA must climb over energy barriers toescape false traps, QA can penetrate these barriers with-out any increase in energy. This e↵ect is a hallmark ofquantum mechanics, known as quantum tunneling. The
standard time-dependent Hamiltonian used for QA is
H(t) = �A(t)NXj=1
�x
j
+ B(t)HP
, (2)
where HP
is written as in Eq. (1) but with the spin vari-ables s
j
replaced with �z
j
Pauli matrices acting on qubitj, and the functions A(t) and B(t) define the anneal-ing schedule parameterized in terms of time t 2 [0, T
QA
](see Fig. 7). These annealing schedules can be defined inmany di↵erent ways as long as the functions are smooth,A(0) ⌧ B(0) and A(T
QA
) � B(TQA
). At the beginningof the annealing, the transverse field magnitude A(t) islarge, and the system dynamics are dominated by quan-tum fluctuations due to tunneling (as opposed to thethermal fluctuations used in SA).
The question of whether D-Wave processors realizecomputationally relevant quantum tunneling has beena subject of substantial debate. This debate has nowbeen settled in the a�rmative with a sequence of publica-tions [6, 8, 9, 12, 13, 18, 21] demonstrating that quantumresources are present and functional in the processors. Inparticular, Refs. [35, 36] studied the performance of theD-Wave device on problems where eight [37] qubit co-tunneling events were employed in a functional mannerto reach low-lying energy solutions.
In order to investigate the computational value of fi-nite range tunneling in QA, we study the scaling of theexponential dependence of annealing time with the sizeof the tunneling domain D,
TQA
= BQA
e↵D , (3)
arX
iv:1
512.
0220
6v4
[qua
nt-p
h] 2
2 Ja
n 20
16
Weak-Strong Cluster Networks
D-Wave Instance Generator (DWIG)
• Generating D-Wave cases is complicated!
https://github.com/lanl-ansi/dwig
D-Wave Instance
Generator (DWIG)
D-Wave Instance Generator (DWIG)
• Generating D-Wave cases is complicated!• Problem Classes
• RAN, RANF (50 loc) • Frustrated Loops (225 loc) • Frustrated Cluster Loops (225 loc) • Weak-Strong Cluster Networks (250 loc)
https://github.com/lanl-ansi/dwig
D-Wave Instance
Generator (DWIG)
D-Wave Instance Generator (DWIG)
• Generating D-Wave cases is complicated!• Problem Classes
• RAN, RANF (50 loc) • Frustrated Loops (225 loc) • Frustrated Cluster Loops (225 loc) • Weak-Strong Cluster Networks (250 loc)
https://github.com/lanl-ansi/dwig
dwig.py ran > ran1.json
D-Wave Instance
Generator (DWIG)
D-Wave Centric Approach to Benchmarking
or-tools
Problem Generation
BQP Test Case
BQPSolvers
DWIG
BQPJSON
And Now This
BQPSolvers
• At some point, you actually want to do some optimization…
https://github.com/lanl-ansi/bqpsolversor-tools
HFS
BQPSolvers
• At some point, you actually want to do some optimization…• What is a high quality solution to my D-Wave problem?
https://github.com/lanl-ansi/bqpsolversor-tools
HFS
BQPSolvers
• At some point, you actually want to do some optimization…• What is a high quality solution to my D-Wave problem?• BQPSolvers makes it easy
• if your data is in BQPJSON
https://github.com/lanl-ansi/bqpsolversor-tools
HFS
BQPSolvers
• At some point, you actually want to do some optimization…• What is a high quality solution to my D-Wave problem?• BQPSolvers makes it easy
• if your data is in BQPJSON
https://github.com/lanl-ansi/bqpsolversor-tools
HFS
dwig.py ran | spin2bool > ran1.json
BQPSolvers
• At some point, you actually want to do some optimization…• What is a high quality solution to my D-Wave problem?• BQPSolvers makes it easy
• if your data is in BQPJSON
https://github.com/lanl-ansi/bqpsolversor-tools
HFS
lns_hfs.py -f ran1.json mip_gurobi.py -f ran1.json mip_cplex.py -f ran1.json bop_ortools.py -f ran1.json
dwig.py ran | spin2bool > ran1.json
The Value of BQPJSON and Friends
Open Source BQP Toolchain
Some Problem SAPI
Open Source BQP Toolchain
Some Problem SAPI
HFS
or-tools
BQPJSOND-Wave Instance
Generator (DWIG)
BQPSOLVERS
qbsolv
problem spec. glue code
Open Source BQP Toolchain
Some Problem SAPI
HFS
or-tools
BQPJSOND-Wave Instance
Generator (DWIG)
BQPSOLVERS
qbsolv
problem spec. glue code
Open Source BQP Toolchain
Some Problem SAPI
HFS
or-tools
BQPJSOND-Wave Instance
Generator (DWIG)
BQPSOLVERS
qbsolv
problem spec. glue code
Everything in this workflow is open source Please extend for your needs and contribute back to the community
The BQP Toolchain in Action!
Baseline Benchmarking Study at the poster sessionRAN Steps vs Runtime
Runtime (seconds)
Freq
uenc
y
0 100 200 300 400 500 6000
5010
015
020
0 k = 1k = 2k = 3k = 4k = 5k = 6k = 7k = 8k = 9k = 10
1 3 5 7 9 12 15 18 22 170 330 410 460 570
Runtime (seconds)
Freq
uenc
y (n
orm
alize
d)
0.0
0.5
1.0
1.5
fcl (n=8347)fl (n=6944)ran (n=1250)ranf (n=1250)wscn (n=30250)
Final Request
For R&D lets develop tools around a standard format
Final Request
For R&D lets develop tools around a standard format
BQPJSON maybe a reasonable choice…
Los Alamos National Laboratory
Questions?
Lets discuss details at the poster session!