Superforecasting by Philip E. Tetlock and Dan Gardner

Embed Size (px)

Citation preview

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    1/29

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    2/29

    http://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136693/siteID/8001/retailerid/6/trackingcode/penguinrandomhttp://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136709/siteID/8001/retailerid/22/trackingcode/PRH698C2DEE54http://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136693/siteID/8001/retailerid/2/trackingcode/PRH698C2DEE54http://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136709/siteID/8001/retailerid/3/trackingcode/PRH698C2DEE54http://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136693/siteID/8001/retailerid/7/trackingcode/randohouseinc10663-20http://www.crownpublishing.com/

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    3/29

    p r f ` s b j h r

    ^ R Q B Y J H Y B N F ^ W @ O K

    —^upbrjhrbnfst`ok  `s f rfrb ihhm teft w`dd lfmb yhu slfrtbr fog w`sbr.

    Hob hj teb k`fots hj ibefv`hrfd sn`bonb rbvbfds ehw th `lprhvb ft

    prbg`nt`ok teb juturb.“

    ‗ FGFL KYF OW,

    Obw Phrm W`lbs  ibstsbdd`ok futehr hj K`vb fog Wfmb

    —Khhg cugklbot fog khhg jhrbnfst`ok frb rfrb, iut teby turo hut

    th ib lfgb hj tbfnefidb sm`dds. Iy jhrn`ok jhrbnfstbrs th nhlpbtb,

    Wbtdhnm g`snhvbrbg weft teb sm`dds frb fog ehw teby whrm, fog te`sihhm tbfnebs teb fi`d`ty th foy `otbrbstbg rbfgbr.“

    ‗ ^ WB[F Y W I Y F OG ,

    prbs`gbot, Web Dhok Ohw Jhuogft`ho

    —Qe`d`p Wbtdhnm `s rbohwobg jhr gblhostrft`ok teft lhst bxpbrts

    frb oh ibttbr tefo ―gfrt-terhw`ok lhombys‐ ft prbg`nt`ok bdbnt`hos, wfrs, bnhohl`n nhddfpsbs, fog htebr bvbots. @o e`s ir`dd`fot obw

    ihhm, Wbtdhnm hjjbrs f lune lhrb ehpbjud lbssfkb, ifsbg honb fkf`o

    ho e`s hwo krhuogirbfm`ok rbsbfrne. Eb sehws teft nbrtf`o pbhpdb

    nfo jhrbnfst bvbots w`te fnnurfny lune ibttbr tefo nefonb‗fog

    sh, pbrefps, nfo teb rbst hj us, `j wb bludftb teb nr`t`nfd te`om`ok

    hj tebsb ―supbrjhrbnfstbrs.‐ Web sbdj-blphwbrlbot kborb ghbso‐t kbt

    foy slfrtbr fog lhrb shpe`st`nftbg tefo te`s.“

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    4/29

    —^upbrjhrbnfst`ok   `s teb rfrb ihhm teft `s ihte snehdfrdy fog bokfk-

    `ok. Web dbsshos frb sn`bot`stn, nhlpbdd`ok, fog bohrlhusdy prfn-

    t`nfd. Foyhob weh `s `o teb jhrbnfst`ok ius`obss‗fog teft‐s fdd hj

    us‗sehudg grhp weft teby frb gh`ok fog rbfg `t.“

    ‗ L @ NE F BD C . L F R IH R ^ ^ @ O ,

    ebfg, Kdhifd J`ofon`fd ^trftbk`bs, Nrbg`t ^u`ssb

    —Webrb `so‐t f shn`fd sn`bot`st `o teb whrdg @ fgl`rb lhrb tefo Qe`d

    Wbtdhnm.“

    ‗ W @ L E F Y J H Y G ,

    futehr hj Web Rogbrnhvbr Bnhohl`st 

    —Jrhl teb Hrfndb hj Gbdpe` th lbg`bvfd fstrhdhkbrs th lhgbro hvbr-

    nhostgbot bxpbrts, jhrbnfstbrs efvb ibbo b`tebr gbdugbg hr jrfugu-

    dbot. Jhr teb strst t`lb, ^upbrjhrbnfst`ok  rbvbfds teb sbnrbt hj lfm`ok

    ehobst, rbd`fidb, bjjbnt`vb, usbjud cugklbots fihut teb juturb.“‗ F F Y H O IY H [ O ,

    ne`bj r`sm hjj`nbr, FVY Nfp`tfd Lfofkblbot, fog

    futehr hj Web Qhmbr Jfnb hj [fdd ^trbbt 

    —^hnrftbs efg teb `os`ket `o ―mohw teysbdj,‐ Mfeoblfo gbd`vbrbg teb

    sn`bonb `o We`om`ok, Jfst fog ^dhw, fog ohw Wbtdhnm efs shlbte`ok

     wb nfo fdd fppdy `o ^upbrjhrbnfst`ok.“

    ‗ CR F O D R @ ^ QBY BS ,

    kdhifd ebfg, RI^ Krhup Ybsbfrne

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    5/29

    Nhpyr`ket © 9162 iy Qe`d`p Wbtdhnm Nhosudt`ok, @on., fog Nhoofuket ^trbbt, @on.

     Fdd r`kets rbsbrvbg.

    Quid`sebg `o teb Ro`tbg ^tftbs iy Nrhwo Quid`sebrs,

    fo `lpr`ot hj teb Nrhwo Quid`se`ok Krhup,

    f g`v`s`ho hj Qboku`o Yfoghl Ehusb DDN, Obw Phrm.

     www.nrhwopuid`se`ok.nhl

    NYH[O fog teb Nrhwo nhdhpeho frb rbk`stbrbgtrfgblfrms hj Qboku`o Yfoghl Ehusb DDN.

    D`irfry hj Nhokrbss Nftfdhk`ok-̀ o-Quid`nft`ho Gftf 

    ]N@Q gftf\

    @^IO 0;5-1-5176->::0->

    b@^IO 0;5-1-5176->:;1-0

    Qr`otbg `o teb Ro`tbg ^tftbs hj Flbr`nf 

    @ddustrft`hos iy Chb DbLhoo`br 

     Cfnmbt gbs`ko iy 

    9 7 : 5 61 0 ; 2 > 6

    J`rst Bg`t`ho

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    6/29

    6

     Fo Hpt`l`st`n ^mbpt`n

    [b frb fdd jhrbnfstbrs. [ebo wb te`om fihut nefok`ok chis, kbt-

    t`ok lfrr`bg, iuy`ok f ehlb, lfm`ok fo `ovbstlbot, dfuone-

    `ok f prhgunt, hr rbt`r`ok, wb gbn`gb ifsbg ho ehw wb bxpbnt tebjuturb w`dd uojhdg. Websb bxpbntft`hos frb jhrbnfsts. Hjtbo wb gh hur

    hwo jhrbnfst`ok. Iut webo i`k bvbots efppbo‗lfrmbts nrfse, wfrs

    dhhl, dbfgbrs trblidb‗ wb turo th teb bxpbrts, tehsb `o teb mohw.

     [b dhhm th pbhpdb d`mb Whl Jr`bglfo.

    @j yhu frb f [e`tb Ehusb stfjjbr, yhu l`ket stog e`l `o teb

    Hvfd Hjstnb w`te teb prbs`gbot hj teb Ro`tbg ^tftbs, tfdm`ok fihut

    teb L`ggdb Bfst. @j yhu frb f Jhrtuob 211 NBH, yhu l`ket spht e`l

    `o Gfvhs, neftt`ok `o teb dhuokb w`te ebgkb juog i`dd`hof`rbs fog

    ^fug` pr`onbs. Fog ̀ j yhu gho‐t jrbqubot teb [e`tb Ehusb hr swfomy

    ^w`ss ehtbds, yhu nfo rbfg e`s Obw Phrm W`lbs  nhdulos fog ibstsbdd-

    `ok ihhms teft tbdd yhu weft‐s efppbo`ok ohw, wey, fog weft w`dd

    nhlb obxt.6 L`dd`hos gh.

    D`mb Whl Jr`bglfo, I`dd Jdfnm jhrbnfsts kdhifd bvbots. Iut tebrb`s f dht dbss gblfog jhr e`s `os`kets.

    Jhr ybfrs, I`dd whrmbg jhr teb R^ Gbpfrtlbot hj Fkr`nudturb `o

     Fr`zhof‗—pfrt p`nm-fog-sehvbd whrm, pfrt sprbfgsebbt“‗iut ohw

    eb d`vbs `o Mbfroby, Obirfsmf. I`dd `s f oft`vb Nhroeusmbr. Eb krbw

    up `o Lfg`sho, Obirfsmf, f jfrl thwo webrb e`s pfrbots hwobg fog

    puid`sebg teb  Lfg`sho ^tfr- Lf`d , f obwspfpbr w`te dhts hj sthr`bs

    fihut dhnfd sphrts fog nhuoty jf`rs. Eb wfs f khhg stugbot `o e`ke

    e d g e i e d j ` g j e

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    7/29

    9 supbrjhrbnfst`ok

     Fr`zhof. Eb wfs f`l`ok jhr f QeG `o lfte, iut eb rbfd`zbg `t wfs

    ibyhog e`s fi`d`t`bs‗—@ efg ly ohsb ruiibg `o ly d`l`tft`hos“ `s

    ehw eb puts `t‗fog eb grhppbg hut. @t wfso‐t wfstbg t`lb, ehwbvbr.Ndfssbs `o hro`tehdhky lfgb I`dd fo fv`g i`rg- wftnebr, fog ibnfusb

     Fr`zhof `s f krbft pdfnb th sbb i`rgs, eb g`g stbdgwhrm pfrt-t`lb jhr

    sn`bot`sts, tebo kht f chi w`te teb Gbpfrtlbot hj Fkr`nudturb fog

    stfybg jhr f we`db.

    I`dd `s stjty-stvb fog rbt`rbg, fdtehuke eb sfys `j shlbhob hjjbrbg

    e`l f chi eb whudg nhos`gbr `t. ^h eb efs jrbb t`lb. Fog eb spbogs

    shlb hj `t jhrbnfst`ok.I`dd efs foswbrbg rhukedy terbb euogrbg qubst`hos d`mb —[`dd

    Yuss`f hjstn`fddy foobx fgg`t`hofd Rmrf`o`fo tbrr`thry `o teb obxt

    terbb lhotes8“ fog —@o teb obxt ybfr, w`dd foy nhuotry w`tegrfw

    jrhl teb burhzhob8“ Weby frb qubst`hos teft lfttbr. Fog teby‐rb

    g`jstnudt. Nhrphrft`hos, ifoms, blifss`bs, fog `otbdd`kbonb fkbon`bs

    strukkdb th foswbr sune qubst`hos fdd teb t`lb. —[`dd Ohrte Mhrbf

    gbthoftb f oundbfr gbv`nb ibjhrb teb bog hj te`s ybfr8“ —Ehw lfoy

    fgg`t`hofd nhuotr`bs w`dd rbphrt nfsbs hj teb Bihdf v`rus `o teb obxt

    b`ket lhotes8“ —[`dd @og`f hr Irfz`d ibnhlb f pbrlfobot lblibr hj

    teb RO ^bnur`ty Nhuon`d `o teb obxt twh ybfrs8“ ^hlb hj teb qubs-

    t`hos frb ghwor`ket hisnurb, ft dbfst jhr lhst hj us. —[`dd OFWH

    `ov`tb obw nhuotr`bs th ch`o teb Lblibrse`p Fnt`ho Qdfo (LFQ)

    `o teb obxt o`ob lhotes8“ —[`dd teb Murg`stfo Ybk`hofd Khvbro-lbot ehdg f rbjbrbogul ho oft`hofd `ogbpbogbonb te`s ybfr8“ —@j

    f oho-Ne`obsb tbdbnhlluo`nft`hos strl w`os f nhotrfnt th prhv`gb

    @otbrobt sbrv`nbs `o teb ^efokef` Jrbb Wrfgb Shob `o teb obxt twh

    ybfrs, w`dd Ne`obsb n`t`zbos efvb fnnbss th Jfnbihhm fog/hr Ww`ttbr8“

     [ebo I`dd strst sbbs hob hj tebsb qubst`hos, eb lfy efvb oh ndub ehw

    th foswbr `t. —[eft ho bfrte `s teb ^efokef` Jrbb Wrfgb Shob8“ eb

    lfy te`om. Iut eb ghbs e`s ehlbwhrm. Eb kftebrs jfnts, ifdfonbs

    d e` g d

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    8/29

      Fo Hpt`l`st`n ^mbpt`n >

    sefrb e`s tehukets ho NOO. Eb efs obvbr ibbo `ov`tbg th Gfvhs th

    s`t ho f pfobd w`te Whl Jr`bglfo. Fog teft‐s uojhrtuoftb. Ibnfusb

    I`dd Jdfnm `s f rblfrmfidb jhrbnfstbr. [b mohw teft ibnfusb bfne hobhj I`dd‐s prbg`nt`hos efs ibbo gftbg, rbnhrgbg, fog fssbssbg jhr fnnu-

    rfny iy ̀ ogbpbogbot sn`bot`stn hisbrvbrs. E`s trfnm rbnhrg ̀ s bxnbddbot.

    I`dd `s oht fdhob. Webrb frb tehusfogs hj htebrs foswbr`ok teb

    sflb qubst`hos. Fdd frb vhduotbbrs. Lhst frbo‐t fs khhg fs I`dd, iut

    fihut 9% frb. Weby `ondugb bok`obbrs fog dfwybrs, frt`sts fog sn`bo-

    t`sts, [fdd ^trbbtbrs fog Lf`o ^trbbtbrs, prhjbsshrs fog stugbots. [b

     w`dd lbbt lfoy hj tebl, `ondug`ok f lfteblft`n`fo, f stdllfmbr,fog shlb rbt`rbbs bfkbr th sefrb teb`r uogbrusbg tfdbots. @ nfdd tebl

    supbrjhrbnfstbrs  ibnfusb teft ̀ s weft teby frb. Ybd`fidb bv`gbonb prhvbs

    `t. Bxpdf`o`ok wey teby‐rb sh khhg, fog ehw htebrs nfo dbfro th gh

     weft teby gh, `s ly khfd `o te`s ihhm.

    Ehw hur dhw-prhstdb supbrjhrbnfstbrs nhlpfrb w`te nbrbirfd nb-

    dbir`t`bs d`mb Whl Jr`bglfo `s fo `otr`ku`ok qubst`ho, iut `t nfo‐t ib

    foswbrbg ibnfusb teb fnnurfny hj Jr`bglfo‐s jhrbnfst`ok efs obvbr

    ibbo r`khrhusdy tbstbg. Hj nhursb Jr`bglfo‐s jfos fog nr`t`ns efvb

    hp`o`hos hob wfy hr teb htebr‗—eb of`dbg teb Frfi ^pr`ok“ hr —eb

    snrbwbg up ho teb 911> `ovfs`ho hj @rfq“ hr —eb wfs prbsn`bot ho

    OFWH bxpfos`ho.“ Iut tebrb frb oh efrg jfnts fihut Whl Jr`bg-

    lfo‐s trfnm rbnhrg, cust bogdbss hp`o`hos‗fog hp`o`hos ho hp`o-

    `hos.9

     Fog teft ̀ s ius`obss fs usufd. Bvbry gfy, teb obws lbg`f gbd`vbrjhrbnfsts w`tehut rbphrt`ok, hr bvbo fsm`ok, ehw khhg teb jhrbnfst-

    brs weh lfgb teb jhrbnfsts rbfddy frb. Bvbry gfy, nhrphrft`hos fog

    khvbrolbots pfy jhr jhrbnfsts teft lfy ib prbsn`bot hr whrtedbss hr

    shlbte`ok `o ibtwbbo. Fog bvbry gfy, fdd hj us‗dbfgbrs hj oft`hos,

    nhrphrftb bxbnut`vbs, `ovbsthrs, fog vhtbrs‗lfmb nr`t`nfd gbn`s`hos

    ho teb ifs`s hj jhrbnfsts wehsb qufd`ty `s uomohwo. Ifsbifdd lfo-

    fkbrs whudgo‐t grbfl hj kbtt`ok hut teb nebnmihhm th e`rb f pdfybr

    ` e d ` j ` ` B j

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    9/29

    7 supbrjhrbnfst`ok

    th teb jhrbnfstbrs weh ebdp us lfmb gbn`s`hos teft lfttbr jfr lhrb

    tefo foy ifsbifdd kflb, wb‐rb nhotbot th ib `kohrfot.>

    @o teft d`ket, rbdy`ok ho I`dd Jdfnm‐s jhrbnfsts dhhms qu`tb rbfsho-fidb. @ogbbg, rbdy`ok ho teb jhrbnfsts hj lfoy rbfgbrs hj te`s ihhm

    lfy prhvb qu`tb rbfshofidb, jhr `t turos hut teft jhrbnfst`ok `s oht f

    —yhu efvb `t hr yhu gho‐t“ tfdbot. @t `s f sm`dd teft nfo ib nudt`vftbg.

    We`s ihhm w`dd sehw yhu ehw.

    WEB HOB FIHRW WEB NE@LQ

    @ wfot th sph`d teb chmb, sh @‐dd k`vb fwfy teb puone d`ob3 teb fvbrfkb

    bxpbrt wfs rhukedy fs fnnurftb fs f gfrt-terhw`ok ne`lpfozbb.

     Phu‐vb prhifidy ebfrg teft hob ibjhrb. @t‐s jflhus‗`o shlb n`r-

    ndbs, `ojflhus. @t efs phppbg up `o teb Obw Phrm W`lbs , teb [fdd

    ^trbbt Churofd , teb J`ofon`fd W`lbs , teb Bnhohl`st , fog htebr hutdbts

    frhuog teb whrdg. @t khbs d`mb te`s3 F rbsbfrnebr kftebrbg f i`k krhup

    hj bxpbrts‗fnfgbl`ns, puog`ts, fog teb d`mb‗th lfmb tehusfogs

    hj prbg`nt`hos fihut teb bnhohly, sthnms, bdbnt`hos, wfrs, fog htebr

    `ssubs hj teb gfy. W`lb pfssbg, fog webo teb rbsbfrnebr nebnmbg teb

    fnnurfny hj teb prbg`nt`hos, eb jhuog teft teb fvbrfkb bxpbrt g`g

    fihut fs wbdd fs rfoghl kubss`ok. Bxnbpt teft‐s oht teb puone d`ob

    ibnfusb —rfoghl kubss`ok“ `so‐t juooy. Web puone d`ob `s fihut fgfrt-terhw`ok ne`lpfozbb. Ibnfusb ne`lpfozbbs frb juooy.

    @ fl teft rbsbfrnebr fog jhr f we`db @ g`go‐t l`og teb chmb. Ly

    stugy wfs teb lhst nhlprbebos`vb fssbsslbot hj bxpbrt cugklbot `o

    teb sn`bot`stn d`tbrfturb. @t wfs f dhok sdhk teft thhm fihut twboty ybfrs,

    jrhl 6057 th 9117, fog teb rbsudts wbrb jfr r`nebr fog lhrb nhostrun-

    t`vb tefo teb puone d`ob sukkbstbg. Iut @ g`go‐t l`og teb chmb ibnfusb

    `t rf`sbg fwfrbobss hj ly rbsbfrne (fog, ybs, sn`bot`sts sfvhr teb`r stj-

    ` j j ) F g @ dj e g g e dg —g

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    10/29

      Fo Hpt`l`st`n ^mbpt`n 2

    @ fdsh g`go‐t l`og ibnfusb teb chmb lfmbs f vfd`g ph`ot. Hpbo

    foy obwspfpbr, wftne foy WZ obws sehw, fog yhu stog bxpbrts weh

    jhrbnfst weft‐s nhl`ok. ^hlb frb nfut`hus. Lhrb frb ihdg fog nhost-gbot. F efogjud ndf`l th ib Hdylp`fo v`s`hofr`bs fidb th sbb gbnfgbs

    `oth teb juturb. [`te jbw bxnbpt`hos, teby frb oht `o jrhot hj teb

    nflbrfs ibnfusb teby phssbss foy prhvbo sm`dd ft jhrbnfst`ok. Fnnu-

    rfny ̀ s sbdghl bvbo lbot`hobg. Hdg jhrbnfsts frb d`mb hdg obws‗shho

    jhrkhttbo‗fog puog`ts frb fdlhst obvbr fsmbg th rbnhon`db weft

    teby sf`g w`te weft fntufddy efppbobg. Web hob uogbo`fidb tfdbot

    teft tfdm`ok ebfgs efvb `s teb`r sm`dd ft tbdd`ok f nhlpbdd`ok sthry w`tenhov`nt`ho, fog teft `s bohuke. Lfoy efvb ibnhlb wbfdtey pbggd`ok

    jhrbnfst`ok hj uotbstbg vfdub th nhrphrftb bxbnut`vbs, khvbrolbot hj-

    stn`fds, fog hrg`ofry pbhpdb weh whudg obvbr te`om hj swfddhw`ok

    lbg`n`ob hj uomohwo bjstnfny fog sfjbty iut weh rhut`obdy pfy jhr

    jhrbnfsts teft frb fs gui`hus fs bd`x`rs shdg jrhl teb ifnm hj f wfkho.

    Websb pbhpdb‗fog teb`r nusthlbrs‗gbsbrvb f ougkb `o teb r`is. @

     wfs efppy th sbb ly rbsbfrne usbg th k`vb t̀ th tebl.

    Iut @ rbfd`zbg teft fs whrg hj ly whrm sprbfg, ̀ ts fppfrbot lbfo-

    `ok wfs lutft`ok. [eft ly rbsbfrne efg sehwo wfs teft teb fvbrfkb

    bxpbrt efg ghob d`ttdb ibttbr tefo kubss`ok ho lfoy hj teb phd`t`nfd

    fog bnhohl`n qubst`hos @ efg phsbg. —Lfoy“ ghbs oht bqufd fdd. @t

     wfs bfs`bst th ibft nefonb ho teb sehrtbst-rfokb qubst`hos teft hody

    rbqu`rbg dhhm`ok hob ybfr hut, fog fnnurfny jbdd hjj teb jurtebr hut bx-pbrts tr`bg th jhrbnfst‗fpprhfne`ok teb gfrt-terhw`ok-ne`lpfozbb

    dbvbd terbb th stvb ybfrs hut. Weft wfs fo `lphrtfot stog`ok. @t tbdds

    us shlbte`ok fihut teb d`l`ts hj bxpbrt`sb `o f nhlpdbx whrdg‗fog

    teb d`l`ts ho weft `t l`ket ib phss`idb jhr bvbo supbrjhrbnfstbrs th

    fne`bvb. Iut fs `o teb ne`dgrbo‐s kflb hj —tbdbpehob,“ `o we`ne f

    perfsb `s we`spbrbg th hob ne`dg weh pfssbs `t ho th fohtebr, fog

    sh ho, fog bvbryhob `s sehnmbg ft teb bog th g`snhvbr ehw lune `t

    e e g e d id g ` e dd

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    11/29

    : supbrjhrbnfst`ok

    bxpbrt jhrbnfsts frb usbdbss,“ we`ne `s ohosbosb. ^hlb vfr`ft`hos wbrb

    bvbo nrugbr‗d`mb —bxpbrts mohw oh lhrb tefo ne`lpfozbbs.“ Ly

    rbsbfrne efg ibnhlb f ifnmsthp rbjbrbonb jhr o`e`d`sts weh sbb tebjuturb fs `oebrbotdy uoprbg`ntfidb fog mohw-ohte`ok phpud`sts weh

    `os`st ho prbnbg`ok —bxpbrt“ w`te —sh-nfddbg.“

    ^h @ t`rbg hj teb chmb. Ly rbsbfrne g`g oht supphrt tebsb lhrb

    bxtrblb nhondus`hos, ohr g`g @ jbbd foy fjsto`ty jhr tebl. Whgfy, teft

    `s fdd teb lhrb trub.

    Webrb ̀ s pdboty hj rhhl th stfmb hut rbfshofidb phs`t`hos ibtwbbo

    teb gbiuombrs fog teb gbjbogbrs hj bxpbrts fog teb`r jhrbnfsts. Hoteb hob efog, teb gbiuombrs efvb f ph`ot. Webrb frb sefgy pbggdbrs

    hj qubst`hofidb `os`kets `o teb jhrbnfst`ok lfrmbtpdfnb. Webrb frb

    fdsh d`l`ts th jhrbs`ket teft lfy cust oht ib surlhuotfidb. Hur gbs`rb

    th rbfne `oth teb juturb w`dd fdwfys bxnbbg hur krfsp. Iut gbiuom-

    brs kh thh jfr webo teby g`sl`ss fdd jhrbnfst`ok fs f jhhd‐s brrfog.

    @ ibd`bvb `t `s phss`idb th sbb `oth teb juturb, ft dbfst `o shlb s`tuf-

    t`hos fog th shlb bxtbot, fog teft foy ̀ otbdd`kbot, hpbo-l`ogbg, fog

    efrgwhrm`ok pbrsho nfo nudt`vftb teb rbqu`s`tb sm`dds.

    Nfdd lb fo —hpt`l`st`n smbpt`n.“

    WEB ^MBQW@N

    Wh uogbrstfog teb —smbpt`n“ efdj hj teft dfibd, nhos`gbr f yhuok Wu-

    o`s`fo lfo puse`ok f whhgbo efognfrt dhfgbg w`te jru`ts fog vbk-

    btfidbs ghwo f gusty rhfg th f lfrmbt `o teb Wuo`s`fo thwo hj ^`g`

    Ihuz`g. [ebo teb lfo wfs terbb, e`s jftebr g`bg. Eb supphrts e`s

    jfl`dy iy ihrrhw`ok lhoby th stdd e`s nfrt, ehp`ok th bfro bohuke

    sbdd`ok teb prhgunb th pfy hjj teb gbit fog efvb f d`ttdb dbjt hvbr. @t‐s

    teb sflb kr`og bvbry gfy. Iut te`s lhro`ok, teb phd`nb fpprhfne teb

    g e ‐ ` m e` d i e e ` d g

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    12/29

      Fo Hpt`l`st`n ^mbpt`n ;

    eb efs oh lhoby. F phd`nbwhlfo sdfps e`l fog `osudts e`s gbfg jf-

    tebr. Weby tfmb e`s snfdbs fog e`s nfrt. Web lfo khbs th f thwo hjstnb

    th nhlpdf`o. Eb `s thdg teb hjstn`fd `s iusy `o f lbbt`ok. Eul`d`ftbg,jur`hus, phwbrdbss, teb lfo dbfvbs.

    Eb rbturos w`te jubd. Huts`gb teb thwo hjstnb eb ghusbs e`lsbdj,

    d`kets f lftne, fog iuros.

    Hody teb nhondus`ho hj te`s sthry `s uousufd. Webrb frb nhuotdbss

    phhr strbbt vboghrs `o Wuo`s`f fog fnrhss teb Frfi whrdg. Qhd`nb nhr-

    rupt`ho `s r`jb, fog eul`d`ft`hos d`mb tehsb `o`ntbg ho te`s lfo frb

    f gf`dy hnnurrbonb. Weby lfttbr th oh hob fs`gb jrhl teb phd`nb fogteb`r v`nt`ls.

    Iut te`s pfrt`nudfr eul`d`ft`ho, ho Gbnblibr 6;, 9161, nfusbg

    Lheflbg Ihufz`z`, fkbg twboty-s`x, th sbt e`lsbdj ho strb, fog

    Ihufz`z`‐s sbdj-`llhdft`ho spfrmbg prhtbsts. Web phd`nb rbsphogbg

     w`te typ`nfd irutfd`ty. Web prhtbsts sprbfg. Ehp`ok th fssufkb teb

    puid`n, teb g`ntfthr hj Wuo`s`f, Qrbs`gbot S`ob bd- Fi`g`ob Ibo Fd`,

    v`s`tbg Ihufz`z` `o teb ehsp`tfd.

    Ihufz`z` g`bg ho Cfoufry 7, 9166. Web uorbst krbw. Ho Cfoufry

    67, Ibo Fd` bg th f nusey bx`db `o ^fug` Frfi`f, bog`ok e`s twboty-

    terbb-ybfr mdbpthnrfny.

    Web Frfi whrdg wftnebg, stuoobg. Webo prhtbsts bruptbg `o

    Bkypt, D`iyf, ^yr`f, Chrgfo, Muwf`t, fog Iferf`o. Fjtbr terbb gb-

    nfgbs `o phwbr, teb Bkypt`fo g`ntfthr Ehso` Luifrfm wfs gr`vbojrhl hjstnb. Bdsbwebrb, prhtbsts swbddbg `oth rbibdd`hos, rbibdd`hos `oth

    n`v`d wfrs. We`s wfs teb Frfi ^pr`ok‗fog `t stfrtbg w`te hob phhr

    lfo, oh g`jjbrbot jrhl nhuotdbss htebrs, ib`ok efrfssbg iy phd`nb, fs

    sh lfoy efvb ibbo, ibjhrb fog s`onb, w`te oh fppfrbot r`ppdb bjjbnts.

    @t `s hob te`ok th dhhm ifnmwfrg fog smbtne f ofrrft`vb frn, fs @

    g`g ebrb, nhoobnt`ok Lheflbg Ihufz`z` th fdd teb bvbots teft hwbg

    hut hj e`s dhobdy prhtbst. Whl Jr`bglfo, d`mb lfoy bd`tb puog`ts, `s

    m`dd g e j ` ` d d ` e L`ggd B

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    13/29

    5 supbrjhrbnfst`ok

    Obw Phrm W`lbs   nhrrbsphogbot `o Dbifoho. Iut nhudg bvbo Whl

    Jr`bglfo, `j eb efg ibbo prbsbot teft jftfd lhro`ok, efvb pbbrbg

    `oth teb juturb fog jhrbsbbo teb sbdj-`llhdft`ho, teb uorbst, teb thp-pd`ok hj teb Wuo`s`fo g`ntfthr, fog fdd teft jhddhwbg8 Hj nhursb oht.

    Oh hob nhudg. Lfyib, k`vbo ehw lune Jr`bglfo mobw fihut teb

    rbk`ho, eb whudg efvb lusbg teft phvbrty fog uoblpdhylbot wbrb

    e`ke, teb oulibr hj gbspbrftb yhuok pbhpdb wfs krhw`ok, nhrrup-

    t`ho wfs rflpfot, rbprbss`ho wfs rbdbotdbss, fog tebrbjhrb Wuo`s`f

    fog htebr Frfi nhuotr`bs wbrb phwgbr mbks wf`t`ok th idhw. Iut

    fo hisbrvbr nhudg efvb grfwo bxfntdy teb sflb nhondus`ho teb ybfribjhrb. Fog teb ybfr ibjhrb teft. @ogbbg, yhu nhudg efvb sf`g teft

    fihut Wuo`s`f, Bkypt, fog sbvbrfd htebr nhuotr`bs jhr gbnfgbs. Weby

    lfy efvb ibbo phwgbr mbks iut teby obvbr idbw‗uot`d Gbnbl-

    ibr 6;, 9161, webo teb phd`nb pusebg teft hob phhr lfo thh jfr.

    @o 60;9 teb Flbr`nfo lbtbhrhdhk`st Bgwfrg Dhrboz wrhtb

    f pfpbr w`te fo frrbst`ok t`tdb3 —Qrbg`ntfi`d`ty3 Ghbs teb Jdfp hj f

    Iuttbry‐s [`oks `o Irfz`d ^bt Hjj f Whrofgh `o Wbxfs8“ F gbnfgb

    bfrd`br, Dhrboz efg g`snhvbrbg iy fnn`gbot teft t`oy gftf botry vfr`f-

    t`hos `o nhlputbr s`ludft`hos hj wbftebr pfttbros‗d`mb rbpdfn`ok

    1.21:69; w`te 1.21:‗nhudg prhgunb grflft`nfddy g`jjbrbot dhok-

    tbrl jhrbnfsts. @t wfs fo ̀ os`ket teft whudg ̀ osp`rb —nefhs tebhry“3 ̀ o

    ohod`obfr systbls d`mb teb ftlhspebrb, bvbo slfdd nefokbs `o `o`t`fd

    nhog`t`hos nfo luserhhl th bohrlhus prhphrt`hos. ^h, `o pr`on`pdb,f dhob iuttbry `o Irfz`d nhudg fp `ts w`oks fog sbt hjj f throfgh `o

    Wbxfs‗bvbo tehuke swfrls hj htebr Irfz`d`fo iuttbr`bs nhudg fp

    jrfot`nfddy teb`r wehdb d`vbs fog obvbr nfusb f oht`nbfidb kust f jbw

    l`dbs fwfy. Hj nhursb Dhrboz g`go‐t lbfo teft teb iuttbry —nfusbs“

    teb throfgh `o teb sflb sbosb teft @ nfusb f w`obkdfss th irbfm webo

    @ e`t `t w`te f efllbr. Eb lbfot teft `j teft pfrt`nudfr iuttbry

    efgo‐t fppbg `ts w`oks ft teft lhlbot, teb uojftehlfidy nhlpdbx

    m j e ` ` g ` dg e i e g

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    14/29

      Fo Hpt`l`st`n ^mbpt`n 0

     Frfi ^pr`ok l`ket obvbr efvb efppbobg, ft dbfst oht webo fog fs `t

    g`g, `j teb phd`nb efg cust dbt Lheflbg Ihufz`z` sbdd e`s jru`ts fog

    vbkbtfidbs teft lhro`ok `o 9161.Bgwfrg Dhrboz se`jtbg sn`bot`stn hp`o`ho thwfrg teb v`bw teft

    tebrb frb efrg d`l`ts ho prbg`ntfi`d`ty, f gbbpdy pe`dhshpe`nfd qubs-

    t`ho.7  Jhr nbotur`bs, sn`bot`sts efg supphsbg teft krhw`ok mohwd-

    bgkb lust dbfg th krbftbr prbg`ntfi`d`ty ibnfusb rbfd`ty wfs d`mb f

    ndhnm‗fo fwbshlbdy i`k fog nhlpd`nftbg ndhnm iut st`dd f ndhnm‗

    fog teb lhrb sn`bot`sts dbfrobg fihut `ts `oofrgs, ehw teb kbfrs

    kr`og thkbtebr, ehw teb wb`kets fog spr`oks juont`ho, teb ibttbr tebynhudg nfpturb `ts hpbrft`hos w`te gbtbrl`o`st`n bquft`hos fog prbg`nt

     weft t̀ whudg gh. @o 6567 teb Jrbone lfteblft`n`fo fog fstrhohlbr

    Q`brrb-^`lho Dfpdfnb thhm te`s grbfl th `ts dhk`nfd bxtrblb3

    [b lfy rbkfrg teb prbsbot stftb hj teb uo`vbrsb fs teb bjjbnt

    hj `ts pfst fog teb nfusb hj `ts juturb. Fo `otbddbnt we`ne ft f

    nbrtf`o lhlbot whudg mohw fdd jhrnbs teft sbt ofturb `o lh-

    t`ho, fog fdd phs`t`hos hj fdd `tbls hj we`ne ofturb `s nhlphsbg,

    `j te`s `otbddbnt wbrb fdsh vfst bohuke th suil`t tebsb gftf th

    fofdys`s, `t whudg blirfnb `o f s`okdb jhrludf teb lhvblbots

    hj teb krbftbst ihg`bs hj teb uo`vbrsb fog tehsb hj teb t`o`bst

    fthl= jhr sune fo `otbddbnt ohte`ok whudg ib uonbrtf`o fog teb

     juturb cust d`mb teb pfst whudg ib prbsbot ibjhrb `ts bybs.

    Dfpdfnb nfddbg e`s `lfk`ofry bot`ty f —gblho.“ @j `t mobw bvbry-

    te`ok fihut teb prbsbot, Dfpdfnb tehuket, t̀ nhudg prbg`nt bvbryte`ok

    fihut teb juturb. @t whudg ib hlo`sn`bot.2

    Dhrboz phurbg nhdg rf`owftbr ho teft grbfl. @j teb ndhnm syl-

    ihd`zbs pbrjbnt Dfpdfnbfo prbg`ntfi`d`ty, `ts hpphs t̀b `s teb Dhrboz`fo

    ndhug. E`ke snehhd sn`bonb tbdds us teft ndhugs jhrl webo wftbr

    d g g ` d We` g ` d i

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    15/29

    61 supbrjhrbnfst`ok

    ho nhlpdbx jbbgifnm `otbrfnt`hos flhok grhpdbts. Wh nfpturb tebsb

    `otbrfnt`hos, nhlputbr lhgbdbrs obbg bquft`hos teft frb e`kedy sbo-

    s`t`vb th t`oy iuttbry-bjjbnt brrhrs `o gftf nhddbnt`ho. ^h bvbo `j wbdbfro fdd teft `s mohwfidb fihut ehw ndhugs jhrl, wb w`dd oht ib fidb

    th prbg`nt teb sefpb f pfrt`nudfr ndhug w`dd tfmb. [b nfo hody wf`t

    fog sbb. @o hob hj e`sthry‐s krbft `rho`bs, sn`bot`sts thgfy mohw vfstdy

    lhrb tefo teb`r nhddbfkubs f nbotury fkh, fog phssbss vfstdy lhrb

    gftf-nruone`ok phwbr, iut teby frb lune dbss nhostgbot `o teb prhs-

    pbnts jhr pbrjbnt prbg`ntfi`d`ty.

    We`s `s f i`k rbfsho jhr teb —smbpt`n“ efdj hj ly —hpt`l`st`n smbp-t`n“ stfonb. [b d`vb `o f whrdg webrb teb fnt`hos hj hob obfrdy phw-

    brdbss lfo nfo efvb r`ppdb bjjbnts frhuog teb whrdg‗r`ppdbs teft

    fjjbnt us fdd th vfry`ok gbkrbbs. F whlfo d`v`ok `o f Mfosfs N`ty

    suiuri lfy te`om Wuo`s`f `s fohtebr pdfobt, fog ebr d`jb efs oh nho-

    obnt`ho th `t, iut `j seb wbrb lfrr`bg th fo f`r jhrnb ofv`kfthr weh

    `bs hut hj teb obfriy [e`tblfo F`r Jhrnb Ifsb, seb l`ket ib sur-

    pr`sbg th dbfro teft hob hisnurb Wuo`s`fo‐s fnt`hos dbg th prhtbsts,

    teft dbg th r`hts, teft dbg th teb thppd`ok hj f g`ntfthr, teft dbg th

    prhtbsts `o D`iyf, teft dbg th f n`v`d wfr, teft dbg th teb 9169 OFWH

    `otbrvbot`ho, teft dbg th ebr eusifog ghgk`ok fot`f`rnrfjt strb hvbr

    Wr`phd`. Weft‐s fo bfs`dy trfnbfidb nhoobnt`ho. Hjtbo teb nhoobnt`hos

    frb efrgbr th spht, iut teby frb fdd frhuog us, `o te`oks d`mb teb pr`nb

     wb pfy ft teb kfs stft`ho hr teb dfyhjjs ghwo teb strbbt. @o f whrdg webrb f iuttbry `o Irfz`d nfo lfmb teb g`jjbrbonb ibtwbbo cust fo-

    htebr suooy gfy `o Wbxfs fog f throfgh tbfr`ok terhuke f thwo, `t‐s

    l`sku`gbg th te`om foyhob nfo sbb vbry jfr `oth teb juturb.:

    WEB HQW@L@^W

    I ` ` e` ` e d` ` g` i`d` g `

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    16/29

      Fo Hpt`l`st`n ^mbpt`n 66

    Nrfom up teb l`nrhsnhpb ho f gfy `o teb d`jb hj teft whlfo

    d`v`ok `o f Mfosfs N`ty suiuri3 Ft :3>1 `o teb lhro`ok, seb grhps

    pfpbrs `oth f ir`bjnfsb, kbts `o ebr nfr, gr`vbs ebr usufd rhutb th whrm,fog pfrms ghwothwo. Fs seb ghbs bvbry wbbmgfy lhro`ok, seb wfdms

    pfst teb stftubs hj d`hos fog `oth teb Krbbm-`osp`rbg hjstnb iu`dg`ok

    hj teb Mfosfs N`ty D`jb @osurfonb Nhlpfoy. Ft ebr gbsm, seb whrms

    ho sprbfgsebbts jhr f we`db, pfrt`n`pftbs `o f nhojbrbonb nfdd ft 613>1,

    spbogs f jbw l`outbs irhws`ok ho teb Flfzho wbis`tb, fog foswbrs

    b-lf`ds uot`d 66321. Webo seb wfdms th f d`ttdb @tfd`fo rbstfurfot th

    efvb duone w`te ebr s`stbr.We`s whlfo‐s d`jb `s `oubonbg iy lfoy uoprbg`ntfidb jfnthrs‗

    jrhl teb dhttbry t`nmbt `o ebr pursb th teb Frfi ^pr`ok teft rbsudts `o

    ebr eusifog y`ok l`ss`hos hvbr D`iyf th teb jfnt teft teb pr`nb hj

    kfs cust wbot up stvb nbots f kfddho ibnfusb tebrb wfs f nhup `o shlb

    nhuotry seb‐s obvbr ebfrg hj‗iut tebrb `s fs lune hr lhrb teft `s

    qu`tb prbg`ntfidb. [ey g`g seb dbfvb ehlb ft :3>18 ^eb g`go‐t wfot

    th kbt stunm `o ruse ehur. Hr th put teft fohtebr wfy, seb prbg`ntbg

    teft trfjstn whudg ib lune ebfv`br dftbr‗fog seb wfs fdlhst nbr-

    tf`ody r`ket ibnfusb ruse ehur `s e`kedy prbg`ntfidb. [ebo seb grhvb,

    seb fot`n`pftbg htebr gr`vbrs‐ ibefv`hr nhostfotdy3 teby w`dd sthp ft

    teb `otbrsbnt`ho webo teb d`ket `s rbg= teby w`dd stfy `o teb`r dfobs

    fog s`kofd ibjhrb turo`ok. ^eb bxpbntbg teb pbhpdb weh sf`g teby

     whudg ch`o teb 613>1 nhojbrbonb nfdd th gh sh, fog seb wfs r`ket.^eb frrfokbg th lbbt ebr s`stbr ft ohho ft teb rbstfurfot ibnfusb teb

    rbstfurfot‐s phstbg ehurs `og`nftbg `t whudg hpbo tebo, fog phstbg

    ehurs frb f rbd`fidb ku`gb.

     [b lfmb luogfob prbg`nt`hos d`mb tebsb rhut`obdy, we`db htebrs

     cust fs rhut`obdy lfmb prbg`nt`hos teft sefpb hur d`vbs. [ebo teb

     whlfo turobg ebr nhlputbr ho, seb `onrbfsbg bdbntr`n`ty nhosulp-

    t`ho ̀ o Mfosfs N`ty iy f i`t, fs bvbry htebr whrmbr ibb g`g teft lhro-

    ` g dd ` d e g g g e g

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    17/29

    69 supbrjhrbnfst`ok

    prhidbls ibnfusb bdbntr`n`ty prhgunbrs fot`n`pftb tebsb surkbs fog

    vfry teb`r hutput fnnhrg`okdy. [ebo teb whlfo wbot th Flfzho,

    teb wbis`tb e`ked`ketbg nbrtf`o prhgunts `t tehuket seb whudg d`mb, fjhrbnfst gbr`vbg jrhl ebr pfst purnefsbs fog irhws`ok fog teft hj

    l`dd`hos hj htebrs. [b nhostfotdy bonhuotbr prbg`nt`vb hpbrft`hos d`mb

    teft ho teb @otbrobt‗Khhkdb pbrshofd`zbs sbfrne rbsudts iy putt`ok

     weft `t te`oms yhu w`dd stog lhst `otbrbst`ok ho thp‗iut teby hpbr-

    ftb sh slhhtedy wb rfrbdy oht`nb. Fog tebo tebrb‐s teb whlfo‐s whrm-

    pdfnb. Web Mfosfs N`ty D`jb @osurfonb Nhlpfoy `s `o teb ius`obss hj

    jhrbnfst`ok g`sfi`d`ty fog gbfte, fog `t ghbs f khhg chi. Weft ghbso‐tlbfo `t mohws prbn`sbdy webo @ w`dd g`b, iut `t ghbs efvb f khhg `gbf

    hj ehw dhok shlbhob hj ly fkb fog prhstdb‗sbx, ̀ onhlb, d`jbstydb‗`s

    d`mbdy th d`vb. Mfosfs N`ty D`jb wfs jhuogbg `o 6502. @j `ts fntufr`bs

     wbrbo‐t khhg jhrbnfstbrs, t̀ whudg efvb khob ifomrupt dhok fkh.

    ^h lune hj hur rbfd`ty `s te`s prbg`ntfidb, hr lhrb sh. @ cust

    Khhkdbg thlhrrhw‐s suor`sb fog suosbt t`lbs jhr Mfosfs N`ty, L`s-

    shur`, fog kht tebl ghwo th teb l`outb. Wehsb jhrbnfsts frb rbd`fidb,

     webtebr teby frb jhr thlhrrhw, teb gfy fjtbr, hr stjty ybfrs jrhl ohw.

    Web sflb `s trub hj t`gbs, bnd`psbs, fog pefsbs hj teb lhho. Fdd nfo

    ib prbg`ntbg jrhl ndhnmd`mb sn`bot`stn dfws w`te bohuke prbn`s`ho th

    sft`sjy Dfpdfnb‐s jhrbnfst`ok gblho.

    Hj nhursb bfne hj tebsb phnmbts hj prbg`ntfi`d`ty nfo ib firuptdy

    puonturbg. F khhg rbstfurfot `s vbry d`mbdy th hpbo `ts ghhrs webo `tsfys `t w`dd, iut `t lfy oht, jhr foy oulibr hj rbfshos, jrhl f lfo-

    fkbr sdbbp`ok dftb, th strb, ifomruptny, pfogbl`n, oundbfr wfr, hr f

    peys`ns bxpbr`lbot fnn`gbotfddy nrbft`ok f idfnm ehdb teft sunms up

    teb shdfr systbl. Web sflb `s trub hj foyte`ok bdsb. Bvbo tehsb stjty-

    ybfr suor`sb fog suosbt jhrbnfsts nhudg ib hjj shlbweft `j, shlb-

    t`lb `o teb obxt stjty ybfrs, f lfss`vb spfnb rhnm iulps Bfrte hjj

    `ts hri`t frhuog teb suo. Webrb frb oh nbrtf`ot`bs `o d`jb‗oht bvbo

    g e g `j ` i i`d` e ` `

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    18/29

      Fo Hpt`l`st`n ^mbpt`n 6>

    f ndhug-nhlput`ok obtwhrm fog teb blbrkbonb hj f juturb shn`bty

    sh puid`n-sp`r`tbg fog prhspbrhus teft teb stftb nfo ib juogbg w`te

    nefr`tfidb ghoft`hos.^h `s rbfd`ty ndhnmd`mb hr ndhug-d`mb8 @s teb juturb prbg`ntfidb hr

    oht8 Websb frb jfdsb g`nehthl`bs, teb strst hj lfoy wb w`dd bonhuotbr.

     [b d`vb `o f whrdg hj ndhnms fog ndhugs fog f vfst culidb hj htebr

    lbtfpehrs. Roprbg`ntfi`d`ty fog prbg`ntfi`d`ty nhbx`st uobfs`dy `o

    teb `otr`nftbdy `otbrdhnm`ok systbls teft lfmb up hur ihg`bs, hur

    shn`bt`bs, fog teb nhslhs. Ehw prbg`ntfidb shlbte`ok `s gbpbogs ho

     weft wb frb try`ok th prbg`nt, ehw jfr `oth teb juturb, fog uogbr weft n`rnulstfonbs.

    Dhhm ft Bgwfrg Dhrboz‐s stbdg. [bftebr jhrbnfsts frb typ`nfddy

    qu`tb rbd`fidb, uogbr lhst nhog`t`hos, dhhm`ok f jbw gfys febfg,

    iut teby ibnhlb `onrbfs`okdy dbss fnnurftb terbb, jhur, fog stvb

    gfys hut. Lune ibyhog f wbbm, wb l`ket fs wbdd nhosudt teft gfrt-

    terhw`ok ne`lpfozbb. ^h wb nfo‐t sfy teft wbftebr `s prbg`ntfidb

    hr oht, hody teft wbftebr `s prbg`ntfidb th shlb bxtbot uogbr shlb

    n`rnulstfonbs‗fog wb lust ib vbry nfrbjud webo wb try th ib lhrb

    prbn`sb tefo teft. Wfmb shlbte`ok fs sbbl`okdy s`lpdb fs teb rbdft`ho-

    se`p ibtwbbo t`lb fog prbg`ntfi`d`ty3 `t `s kbobrfddy trub teft teb jur-

    tebr wb try th dhhm ̀ oth teb juturb, teb efrgbr ̀ t ̀ s th sbb. Iut tebrb nfo

    ib prhdhokbg bxnbpt`hos th teb rudb. Qrbg`nt`ok teb nhot`ouft`ho hj f

    dhok iudd lfrmbt ̀ o sthnms nfo prhvb prhsttfidb jhr lfoy ybfrs‗uot`d`t suggbody prhvbs th ib yhur uogh`ok. Fog prbg`nt`ok teft g`ohsfurs

     whudg nhot`oub th prbs`gb ft teb thp hj teb jhhg nef`o wfs f sfjb ibt

    jhr tbos hj l`dd`hos hj ybfrs‗uot`d fo fstbrh`g sbt hjj f nftfndysl

    teft hpbobg up bnhdhk`nfd o`nebs jhr f t`oy lfllfd teft bvbotufddy

    bvhdvbg `oth f spbn`bs teft tr`bs th prbg`nt teb juturb. Dfws hj peys`ns

    fs`gb, tebrb frb oh uo`vbrsfd nhostfots, sh sbpfrft`ok teb prbg`ntfidb

    jrhl teb uoprbg`ntfidb ̀ s g`jstnudt whrm. Webrb‐s oh wfy frhuog ̀ t.

    L d ` m e i e We m d

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    19/29

    67 supbrjhrbnfst`ok

     wey wb mohw teft hob- fog twh-gfy jhrbnfsts frb typ`nfddy qu`tb fn-

    nurftb we`db b`ket-gfy jhrbnfsts frb oht. [`te tebsb fofdysbs, lbtbh-

    rhdhk`sts frb fidb th sefrpbo teb`r uogbrstfog`ok hj ehw wbftebr whrmsfog twbfm teb`r lhgbds. Webo teby try fkf`o. Jhrbnfst, lbfsurb, rb-

    v`sb. Ybpbft. @t‐s f obvbr-bog`ok prhnbss hj `onrblbotfd `lprhvblbot

    teft bxpdf`os wey wbftebr jhrbnfsts frb khhg fog sdhwdy kbtt`ok ibttbr.

    Webrb lfy ib d`l`ts th sune `lprhvblbots, ehwbvbr, ibnfusb wbftebr

    `s teb tbxtihhm `ddustrft`ho hj ohod`obfr`ty. Web jurtebr hut teb jhrb-

    nfstbr tr`bs th dhhm, teb lhrb hpphrtuo`ty tebrb `s jhr nefhs th fp `ts

    iuttbry w`oks fog idhw fwfy bxpbntft`hos. I`k dbfps `o nhlput`okphwbr fog nhot`oubg rbstoblbot hj jhrbnfst`ok lhgbds lfy ougkb teb

    d`l`ts f d`ttdb jurtebr `oth teb juturb iut tehsb fgvfonbs krfgufddy kbt

    efrgbr fog teb pfyhjjs ser`om thwfrg zbrh. Ehw khhg nfo `t kbt8 Oh

    hob mohws. Iut mohw`ok teb nurrbot d`l`ts `s t̀sbdj f sunnbss.

    @o sh lfoy htebr e`ke-stfmbs bogbfvhrs, jhrbnfstbrs frb krhp`ok

    `o teb gfrm. Weby efvb oh `gbf ehw khhg teb`r jhrbnfsts frb `o teb

    sehrt, lbg`ul, hr dhok tbrl‗fog oh `gbf ehw khhg teb`r jhrbnfsts

    nhudg ibnhlb. Ft ibst, teby efvb vfkub euonebs. Weft‐s ibnfusb teb

    jhrbnfst-lbfsurb-rbv`sb prhnbgurb hpbrftbs hody w`te`o teb rfrbstbg

    nhostobs hj e`ke-tbne jhrbnfst`ok, sune fs teb whrm hj lfnrhbnhoh-

    l`sts ft nbotrfd ifoms hr lfrmbt`ok fog stofon`fd prhjbss`hofds `o

    i`k nhlpfo`bs hr hp`o`ho phdd fofdysts d`mb Oftb ^`dvbr.; Lhrb hjtbo

    jhrbnfsts frb lfgb fog tebo . . . ohte`ok. Fnnurfny `s sbdghl gbtbr-l`obg fjtbr teb jfnt fog `s fdlhst obvbr ghob w`te sujstn`bot rbkudfr-

    `ty fog r`khr teft nhondus`hos nfo ib grfwo. Web rbfsho8 Lhstdy ̀ t‐s f

    gblfog-s`gb prhidbl3 Web nhosulbrs hj jhrbnfst`ok‗khvbrolbots,

    ius`obss, fog teb puid`n‗gho‐t gblfog bv`gbonb hj fnnurfny. ^h

    tebrb `s oh lbfsurblbot. [e`ne lbfos oh rbv`s`ho. Fog w`tehut

    rbv`s`ho, tebrb nfo ib oh `lprhvblbot. @lfk`ob f whrdg `o we`ne

    pbhpdb dhvb th ruo, iut teby efvb oh `gbf ehw jfst teb fvbrfkb pbrsho

    e j e i dg i e

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    20/29

      Fo Hpt`l`st`n ^mbpt`n 62

    teb kuo `s strbg, bog `t fjtbr f spbn`stbg g`stfonb‗fog tebrb frb oh

    `ogbpbogbot rfnb hjstn`fds fog t`lbmbbpbrs lbfsur`ok rbsudts. Ehw

    d`mbdy `s `t teft ruoo`ok t`lbs frb `lprhv`ok `o te`s whrdg8 Oht vbry. Frb teb ibst ruoobrs ruoo`ok fs jfst fs eulfo ib`oks frb peys`nfddy

    nfpfidb8 Fkf`o, prhifidy oht.

    —@ efvb ibbo strunm iy ehw `lphrtfot lbfsurblbot `s th `l-

    prhv`ok teb eulfo nhog`t`ho,“ I`dd Kftbs wrhtb. —Phu nfo fne`bvb

    `onrbg`idb prhkrbss `j yhu sbt f ndbfr khfd fog stog f lbfsurb teft w`dd

    gr`vb prhkrbss thwfrg teft khfd. . . . We`s lfy sbbl ifs`n, iut `t `s

    flfz`ok ehw hjtbo `t `s oht ghob fog ehw efrg `t `s th kbt r`ket.“5 Eb`s r`ket fihut weft `t tfmbs th gr`vb prhkrbss, fog `t `s surpr`s`ok ehw

    rfrbdy `t‐s ghob `o jhrbnfst`ok. Bvbo teft s`lpdb strst stbp‗sbtt`ok f

    ndbfr khfd‗efso‐t ibbo tfmbo.

     Phu l`ket te`om teb khfd hj jhrbnfst`ok `s th jhrbsbb teb juturb

    fnnurftbdy, iut teft‐s hjtbo oht teb khfd, hr ft dbfst oht teb shdb khfd.

    ^hlbt`lbs jhrbnfsts frb lbfot th botbrtf`o. We`om hj NOIN‐s C`l

    Nrflbr w`te e`s —ihhyfe!“ set`nm, hr Cheo LnDfuked`o, teb ehst

    hj Web LnDfuked`o Krhup, ibddhw`ok ft e`s pfobd`sts th prbg`nt teb

    d`mbd`ehhg hj fo bvbot —ho f snfdb jrhl zbrh th tbo, w`te zbrh rbprb-

    sbot`ok zbrh phss`i`d`ty fog tbo rbprbsbot`ok nhlpdbtb lbtfpeys`nfd

    nbrt`tugb!“ ^hlbt`lbs jhrbnfsts frb usbg th fgvfonb phd`t`nfd fkbo-

    gfs fog kfdvfo`zb fnt`ho‗fs fnt`v`sts ehpb th gh webo teby wfro

    hj dhhl`ok ehrrhrs uodbss wb nefokb hur wfys. Webrb `s fdsh grbss-th-`lprbss jhrbnfst`ok‗ we`ne `s weft ifoms gbd`vbr webo teby pfy

    f jflhus puog`t th tbdd wbfdtey nd`bots fihut teb kdhifd bnhohly `o

    9121. Fog shlb jhrbnfsts frb lbfot th nhljhrt‗iy fssur`ok teb

    fug`bonb teft teb`r ibd`bjs frb nhrrbnt fog teb juturb w`dd uojhdg fs

    bxpbntbg. Qfrt`sfos frb jhog hj tebsb jhrbnfsts. Weby frb teb nhko`-

    t`vb bqu`vfdbot hj sd`pp`ok `oth f wfrl ifte.

    We`s culidb hj khfds `s sbdghl fnmohwdbgkbg, we`ne lfmbs `t

    g`jst d m` g g @ ‐

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    21/29

    6: supbrjhrbnfst`ok

     Fog ybt te`s stfkoft`ho `s f i`k rbfsho wey @ fl fo hpt`l`st`n  

    smbpt`n. [b mohw teft `o sh lune hj weft pbhpdb wfot th prbg`nt‗

    phd`t`ns, bnhohl`ns, stofonb, ius`obss, tbneohdhky, gf`dy d`jb‗prbg`nt-fi`d`ty bx`sts, th shlb gbkrbb, `o shlb n`rnulstfonbs. Iut tebrb `s sh

    lune bdsb wb gh oht mohw. Jhr sn`bot`sts, oht mohw`ok ̀ s bxn`t`ok. @t‐s

    fo hpphrtuo`ty th g`snhvbr= teb lhrb teft ̀ s uomohwo, teb krbftbr teb

    hpphrtuo`ty. Wefoms th teb jrfomdy qu`tb flfz`ok dfnm hj r`khr `o sh

    lfoy jhrbnfst`ok ghlf`os, te`s hpphrtuo`ty `s eukb. Fog th sb`zb `t,

    fdd wb efvb th gh ̀ s sbt f ndbfr khfd‗fnnurfny!‗fog kbt sbr`hus fihut

    lbfsur`ok.@‐vb ibbo gh`ok teft jhr lune hj ly nfrbbr. Web rbsbfrne teft

    prhgunbg teb gfrt-terhw`ok-ne`lpfozbb rbsudt wfs pefsb hob. Qefsb

    twh stfrtbg `o teb sullbr hj 9166, webo ly rbsbfrne (fog d`jb)

    pfrtobr Ifrifrf Lbddbrs fog @ dfuonebg teb Khhg Cugklbot Qrhcbnt

    fog `ov`tbg vhduotbbrs th s`ko up fog jhrbnfst teb juturb. I`dd Jdfnm

    rbsphogbg. ^h g`g f nhupdb hj tehusfog htebrs teft strst ybfr, fog

    tehusfogs lhrb `o teb jhur ybfrs teft jhddhwbg. Nuludft`vbdy, lhrb

    tefo twboty tehusfog `otbddbntufddy nur`hus dfypbhpdb tr`bg th stkurb

    hut `j prhtbsts `o Yuss`f whudg sprbfg, teb pr`nb hj khdg whudg pdul-

    lbt, teb O`mmb` whudg ndhsb fihvb 0,211, wfr whudg brupt ho teb

    Mhrbfo pbo`osudf, fog lfoy htebr qubst`hos fihut nhlpdbx, nefd-

    dbok`ok kdhifd `ssubs. Iy vfry`ok teb bxpbr`lbotfd nhog`t`hos, wb

    nhudg kfukb we`ne jfnthrs `lprhvbg jhrbs`ket, iy ehw lune, hvbr we`ne t`lb jrflbs, fog ehw khhg jhrbnfsts nhudg ibnhlb `j ibst

    prfnt`nbs wbrb dfybrbg ho bfne htebr. Df`g hut d`mb teft, `t shuogs

    s`lpdb. @t wfso‐t. @t wfs f gblfog`ok prhkrfl teft thhm teb tfdbots

    fog efrg whrm hj f ludt`g`sn`pd`ofry tbfl ifsbg ft teb Ro`vbrs`ty hj

    Nfd`jhro`f, Ibrmbdby, fog teb Ro`vbrs t̀y hj Qboosydvfo`f.

    I`k fs `t wfs, teb Khhg Cugklbot Qrhcbnt (KCQ) wfs hody pfrt hj

    f lune dfrkbr rbsbfrne bjjhrt sphoshrbg iy teb @otbdd`kbonb Fgvfonbg

    Y e Q c F ` ` (@FYQF) G ‐ i jj i e id g

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    22/29

      Fo Hpt`l`st`n ^mbpt`n 6;

    rbphrts th teb g`rbnthr hj Oft`hofd @otbdd`kbonb fog `ts chi `s th sup-

    phrt gfr`ok rbsbfrne teft prhl`sbs th lfmb Flbr`nfo `otbdd`kbonb

    ibttbr ft weft `t ghbs. Fog f i`k pfrt hj weft Flbr`nfo `otbdd`kbonbghbs `s jhrbnfst kdhifd phd`t`nfd fog bnhohl`n trbogs. Iy hob rhuke

    bst`lftb, teb Ro`tbg ^tftbs efs twboty tehusfog `otbdd`kbonb fofdysts

    fssbss`ok bvbryte`ok jrhl l`outb puzzdbs th lfchr bvbots sune fs

    teb d`mbd`ehhg hj fo @srfbd` sobfm fttfnm ho @rfo`fo oundbfr jfn`d`-

    t`bs hr teb gbpfrturb hj Krbbnb jrhl teb burhzhob.0 Ehw khhg `s fdd

    te`s jhrbnfst`ok8 Weft `s oht bfs`dy foswbrbg ibnfusb teb `otbdd`kbonb

    nhlluo`ty, d`mb sh lfoy lfchr prhgunbrs hj jhrbnfst`ok, efs obvbribbo mbbo ho spbog`ok lhoby th stkurb teft hut. Webrb frb vfr`hus

    rbfshos jhr teft rbduntfonb, shlb lhrb rbspbntfidb tefo htebrs, iut

     wb‐dd kbt `oth teft dftbr. [eft lfttbrs `s teft te`s jhrbnfst`ok `s nr`t`-

    nfd th oft`hofd sbnur`ty fog ybt d`ttdb nfo ib sf`g w`te foy nhostgbonb

    fihut ehw khhg `t `s, hr bvbo webtebr `t‐s fs khhg fs f ludt`i`dd`ho-

    ghddfr hpbrft`ho w`te twboty tehusfog pbhpdb sehudg ib. Wh nefokb

    teft, @FYQF nrbftbg f jhrbnfst`ok thuroflbot `o we`ne stvb sn`bo-

    t`stn tbfls dbg iy thp rbsbfrnebrs `o teb stbdg whudg nhlpbtb th kbo-

    brftb fnnurftb jhrbnfsts ho teb shrts hj thuke qubst`hos `otbdd`kbonb

    fofdysts gbfd w`te bvbry gfy. Web Khhg Cugklbot Qrhcbnt wfs hob

    hj tehsb stvb tbfls. Bfne tbfl whudg bjjbnt`vbdy ib `ts hwo rbsbfrne

    prhcbnt, jrbb th `lprhv`sb weftbvbr lbtehgs `t tehuket whudg whrm,

    iut rbqu`rbg th suil`t jhrbnfsts ft 0 f.l. bfstbro stfogfrg t`lbbvbry gfy jrhl ^bptblibr 9166 th Cuob 9162. Iy rbqu`r`ok tbfls

    th jhrbnfst teb sflb qubst`hos ft teb sflb t`lb, teb thuroflbot nrb-

    ftbg f dbvbd pdfy`ok stbdg‗fog f r`ne trhvb hj gftf fihut weft whrms,

    ehw wbdd, fog webo. Hvbr jhur ybfrs, @FYQF phsbg obfrdy stvb euo-

    grbg qubst`hos fihut whrdg fjjf`rs. W`lb jrflbs wbrb sehrtbr tefo `o

    ly bfrd`br rbsbfrne, w`te teb vfst lfchr`ty hj jhrbnfsts bxtbog`ok hut

    lhrb tefo hob lhote fog dbss tefo hob ybfr. @o fdd, wb kftebrbg hvbr

    `dd` ` g` `g d c g i e j

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    23/29

    65 supbrjhrbnfst`ok

     wb ibft teb nhotrhd krhup iy ;5%. KCQ fdsh ibft `ts uo`vbrs`ty-

    fjstd`ftbg nhlpbt`thrs, `ondug`ok teb Ro`vbrs`ty hj L`ne`kfo fog

    L@W, iy ebjty lfrk`os, jrhl >1% th ;1%, fog bvbo hutpbrjhrlbgprhjbss`hofd `otbdd`kbonb fofdysts w`te fnnbss th ndfss`stbg gftf. Fjtbr

    twh ybfrs, KCQ wfs gh`ok sh lune ibttbr tefo `ts fnfgbl`n nhlpbt`-

    thrs teft @FYQF grhppbg teb htebr tbfls.61

    @‐dd gbdvb `oth gbtf`ds dftbr, iut dbt‐s ohtb twh mby nhondus`hos teft

    blbrkb jrhl te`s rbsbfrne. Hob, jhrbs`ket `s rbfd. ^hlb pbhpdb‗

    pbhpdb d`mb I`dd Jdfnm‗efvb ̀ t ̀ o spfgbs. Weby frbo‐t kurus hr hrfndbs

     w`te teb phwbr th pbbr gbnfgbs `oth teb juturb, iut teby gh efvb frbfd, lbfsurfidb sm`dd ft cugk`ok ehw e`ke-stfmbs bvbots frb d`mbdy th

    uojhdg terbb lhotes, s`x lhotes, f ybfr, hr f ybfr fog f efdj `o fg-

    vfonb. Web htebr nhondus`ho `s weft lfmbs tebsb supbrjhrbnfstbrs sh

    khhg. @t‐s oht rbfddy weh teby frb. @t `s weft teby gh. Jhrbs`ket `so‐t f

    lystbr`hus k`jt ibsthwbg ft i`rte. @t `s teb prhgunt hj pfrt`nudfr wfys

    hj te`om`ok, hj kftebr`ok `ojhrlft`ho, hj upgft`ok ibd`bjs. Websb

    efi`ts hj tehuket nfo ib dbfrobg fog nudt`vftbg iy foy `otbdd`kbot,

    tehuketjud, gbtbrl`obg pbrsho. @t lfy oht bvbo ib fdd teft efrg th

    kbt stfrtbg. Hob rbsudt teft pfrt`nudfrdy surpr`sbg lb wfs teb bjjbnt

    hj f tuthr`fd nhvbr`ok shlb ifs`n nhonbpts teft wb‐dd bxpdhrb `o te`s

    ihhm fog frb sullfr`zbg `o teb Wbo Nhllfoglbots fppbog`x. @t

    thhm hody fihut s`xty l`outbs th rbfg fog ybt `t `lprhvbg fnnurfny

    iy rhukedy 61% terhuke teb bot`rb thuroflbot ybfr. Pbs, 61% lfyshuog lhgbst, iut `t wfs fne`bvbg ft sh d`ttdb nhst. Fog obvbr jhrkbt

    teft bvbo lhgbst `lprhvblbots `o jhrbs`ket lf`otf`obg hvbr t`lb

    fgg up. @ sphmb fihut teft w`te Ffrho Irhwo, fo futehr, f [fdd

    ^trbbt vbtbrfo, fog teb ne`bj r`sm lfofkbr ft FVY Nfp`tfd Lfofkb-

    lbot, f ebgkb juog w`te hvbr $611 i`dd`ho `o fssbts. —@t‐s sh efrg th

    sbb ibnfusb `t‐s oht grflft`n,“ eb sf`g, iut `j `t `s sustf`obg —`t‐s teb

    g`jjbrbonb ibtwbbo f nhos`stbot w`oobr weh‐s lfm`ok f d`v`ok, hr teb

    e ‐ ` i m dd e ` “66 F dg d m d

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    24/29

      Fo Hpt`l`st`n ^mbpt`n 60

     wb`kets fog flftburs, seb sf`g, `s teft teb ebfvywb`kets mohw teb

    g`jjbrbonb ibtwbbo f :1 ⁇ 71 ibt fog f 71 ⁇ :1 ibt.

     Fog ybt, `j `t‐s phss`idb th `lprhvb jhrbs`ket s`lpdy iy lbfsur-`ok, fog `j teb rbwfrgs hj `lprhvbg jhrbs`ket frb suistfot`fd, wey

    `so‐t lbfsur`ok stfogfrg prfnt`nb8 F i`k pfrt hj teb foswbr th teft

    qubst`ho d`bs `o teb psynehdhky teft nhov`onbs us wb mohw te`oks

     wb rbfddy gho‐t‗te`oks d`mb webtebr Whl Jr`bglfo `s fo fnnurftb

    jhrbnfstbr hr oht. @‐dd bxpdhrb te`s psynehdhky `o nefptbr 9. Jhr nbo-

    tur`bs, `t ehiidbg prhkrbss `o lbg`n`ob. [ebo peys`n`fos stofddy fn-

    nbptbg teft teb`r bxpbr`bonb fog pbrnbpt`hos wbrb oht rbd`fidb lbfoshj gbtbrl`o`ok webtebr f trbftlbot whrms, teby turobg th sn`bot`stn

    tbst`ok‗fog lbg`n`ob stofddy stfrtbg th lfmb rfp`g fgvfonbs. Web

    sflb rbvhdut`ho obbgs th efppbo `o jhrbnfst`ok.

    @t who‐t ib bfsy. Nefptbr > bxfl`obs weft `t tfmbs th tbst jhrb-

    nfst`ok fs r`khrhusdy fs lhgbro lbg`n`ob tbsts trbftlbots. @t‐s f i`k-

    kbr nefddbokb tefo `t lfy fppbfr. @o teb dftb 6051s @ whrmbg hut f

    lbtehghdhky fog nhoguntbg weft wfs, ft teb t`lb, teb i`kkbst tbst hj

    bxpbrt phd`t`nfd jhrbnfst`ok fnnurfny bvbr. Hob rbsudt, gbd`vbrbg lfoy

    ybfrs dftbr, wfs teb puone d`ob teft ohw lfmbs lb squ`rl. Iut fo-

    htebr g`snhvbry hj teft rbsbfrne g`go‐t rbnb`vb obfrdy fs lune fttbo-

    t`ho bvbo tehuke `t wfs jfr lhrb `lphrtfot3 hob krhup hj bxpbrts

    efg lhgbst iut rbfd jhrbs`ket. [eft lfgb teb g`jjbrbonb ibtwbbo teb

    bxpbrts w`te jhrbs`ket fog tehsb weh wbrb sh ehpbdbss teby grfkkbgteb fvbrfkb ghwo th teb dbvbd hj f gfrt-terhw`ok ne`lp8 @t wfso‐t

    shlb lyst`nfd k`jt hr fnnbss th `ojhrlft`ho htebrs g`go‐t efvb. Ohr

     wfs `t foy pfrt`nudfr sbt hj ibd`bjs. @ogbbg, w`te`o f qu`tb w`gb rfokb

    hj v`bws, weft  teby tehuket g`go‐t lfttbr. @t wfs ehw  teby tehuket.

    @osp`rbg `o pfrt iy teft `os`ket, @FYQF nrbftbg `ts uoprbnb-

    gbotbg jhrbnfst`ok thuroflbot. Nefptbr 7 `s teb sthry hj ehw teft

    efppbobg‗fog teb g`snhvbry hj supbrjhrbnfstbrs. [ey frb teby sh

    g We ` e e e 2 e e [e

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    25/29

    91 supbrjhrbnfst`ok

    l`ket suspbnt `t‐s `otbdd`kbonb teft lfmbs fdd teb g`jjbrbonb. @t‐s oht.

    Weby‐rb fdsh rblfrmfidy oulbrftb. D`mb I`dd Jdfnm, lfoy efvb fg-

    vfonbg gbkrbbs `o lfteblft`ns fog sn`bonb. ^h `s teb sbnrbt frnfoblfte8 Oh. Bvbo supbrjhrbnfstbrs weh frb nfrg-nfrry`ok lfteblft`-

    n`fos rfrbdy usb lune lfte. Weby fdsh tbog th ib obwscuom`bs weh

    stfy ho thp hj teb dftbst gbvbdhplbots fog rbkudfrdy upgftb teb`r jhrb-

    nfsts, sh yhu l`ket ib tblptbg th fttr`iutb teb`r sunnbss th spbog`ok

    bogdbss ehurs ho teb chi. Pbt teft thh whudg ib f l`stfmb.

    ^upbrjhrbnfst`ok ghbs rbqu`rb l`o`lul dbvbds hj `otbdd`kbonb,

    oulbrfny, fog mohwdbgkb hj teb whrdg, iut foyhob weh rbfgs sbr`-hus ihhms fihut psynehdhk`nfd rbsbfrne prhifidy efs tehsb prbrbq-

    u`s`tbs. ^h weft `s `t teft bdbvftbs jhrbnfst`ok th supbrjhrbnfst`ok8 Fs

     w`te teb bxpbrts weh efg rbfd jhrbs`ket `o ly bfrd`br rbsbfrne, weft

    lfttbrs lhst `s ehw  teb jhrbnfstbr te`oms. @‐dd gbsnr`ib te`s `o gbtf`d,

    iut irhfgdy spbfm`ok, supbrjhrbnfst`ok gblfogs te`om`ok teft `s

    hpbo-l`ogbg, nfrbjud, nur`hus, fog‗fihvb fdd‗sbdj-nr`t`nfd. @t fdsh

    gblfogs jhnus. Web m`og hj te`om`ok teft prhgunbs supbr`hr cugk-

    lbot ghbs oht nhlb bjjhrtdbssdy. Hody teb gbtbrl`obg nfo gbd`vbr `t

    rbfshofidy nhos`stbotdy, we`ne `s wey hur fofdysbs efvb nhos`stbotdy

    jhuog nhll`tlbot th sbdj-̀ lprhvblbot th ib teb strhokbst prbg`nthr

    hj pbrjhrlfonb.

    @o teb stofd nefptbrs, @‐dd rbshdvb fo fppfrbot nhotrfg`nt`ho ib-

    twbbo teb gblfogs hj khhg cugklbot fog bjjbnt`vb dbfgbrse`p,rbsphog th weft @ te`om frb teb twh strhokbst nefddbokbs th ly rb-

    sbfrne, fog nhondugb‗fpprhpr`ftbdy jhr f ihhm fihut jhrbnfst`ok‗

     w`te f nhos`gbrft`ho hj weft nhlbs obxt.

    F JHYBNF^W FIHRW JHYBNF^W@OK

    I i e` m e` ` dd e d d g g Fj dd

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    26/29

      Fo Hpt`l`st`n ^mbpt`n 96

    fdkhr`tels, fog I`k Gftf. Ft `ts nhrb, teb jhrbnfst`ok @ stugy `ovhdvbs

    suicbnt`vb cugklbot3 `t `s pbhpdb te`om`ok fog gbn`g`ok, ohte`ok

    lhrb. @so‐t teb t`lb jhr sune sdhppy kubsswhrm grfw`ok th f ndhsb8@o 6027, f ir`dd`fot psynehdhk`st, Qfud Lbbed wrhtb f slfdd ihhm

    teft nfusbg f i`k st`r.69 @t rbv`bwbg twboty stug`bs sehw`ok teft wbdd-

    `ojhrlbg bxpbrts prbg`nt`ok hutnhlbs‗ webtebr f stugbot whudg

    sunnbbg `o nhddbkb hr f pfrhdbb whudg ib sbot ifnm th pr`sho‗ wbrb

    oht fs fnnurftb fs s`lpdb fdkhr`tels teft fggbg up hicbnt`vb `og`nf-

    thrs d`mb fi`d`ty tbst snhrbs fog rbnhrgs hj pfst nhogunt. Lbbed‐s ndf`l

    upsbt lfoy bxpbrts, iut suisbqubot rbsbfrne‗ohw lhrb tefo twheuogrbg stug`bs‗efs sehwo teft `o lhst nfsbs stft`st`nfd fdkhr`tels

    ibft suicbnt`vb cugklbot, fog `o teb efogjud hj stug`bs webrb teby

    gho‐t, teby usufddy t`b. K`vbo teft fdkhr`tels frb qu`nm fog nebfp,

    uod`mb suicbnt`vb cugklbot, f t`b supphrts us`ok teb fdkhr`tel. Web

    ph`ot `s ohw `og`sputfidb3 webo yhu efvb f wbdd-vfd`gftbg stft`st`nfd

    fdkhr`tel, usb `t .

    We`s `os`ket wfs obvbr f terbft th teb rb`ko hj suicbnt`vb cugk-

    lbot ibnfusb wb sh rfrbdy efvb wbdd-vfd`gftbg fdkhr`tels jhr teb

    prhidbl ft efog. @t wfs cust `lprfnt`nfd jhr lfte th g`spdfnb pdf`o

    hdg te`om`ok‗`o 6027 fog bvbo thgfy.

    Iut spbntfnudfr fgvfonbs `o `ojhrlft`ho tbneohdhky sukkbst wb

    frb fpprhfne`ok f e`sthr`nfd g`snhot`ou`ty `o eulfo`ty‐s rbdft`ho-

    se`p w`te lfne`obs. @o 600; @IL‐s Gbbp Idub ibft nebss neflp`hoKfrry Mfspfrhv. Ohw, nhllbrn`fddy fvf`dfidb nebss prhkrfls nfo

    ibft foy eulfo. @o 9166 @IL‐s [ftsho ibft Cbhpfrgy!  neflp`hos

    Mbo Cboo`oks fog Irfg Yuttbr. Weft wfs f vfstdy thukebr nhlput`ok

    nefddbokb, iut [ftsho‐s bok`obbrs g`g `t. Whgfy, `t‐s oh dhokbr `lphs-

    s`idb th `lfk`ob f jhrbnfst`ok nhlpbt`t`ho `o we`ne f supbrnhlputbr

    trhuonbs supbrjhrbnfstbrs fog supbrpuog`ts fd`mb. Fjtbr teft efp-

    pbos, tebrb w`dd st`dd ib eulfo jhrbnfstbrs, iut d`mb eulfo Cbhpfrgy!  

    `dd d e e j `

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    27/29

    99 supbrjhrbnfst`ok

    teft [ftsho nhudg bfs`dy stbdg f qubst`ho fihut teb prbsbot hr pfst

    d`mb —[e`ne twh Yuss`fo dbfgbrs trfgbg chis `o teb dfst tbo ybfrs8“

    Iut @ wfs nur`hus fihut e`s v`bws ho ehw dhok `t w`dd tfmb jhr [ft-sho hr hob hj `ts g`k`tfd gbsnbogfots th stbdg qubst`hos d`mb —[`dd twh

    thp Yuss`fo dbfgbrs trfgb chis `o teb obxt tbo ybfrs8“

    @o 60:2 teb phdylfte Ebribrt ^`lho tehuket wb wbrb hody

    twboty ybfrs fwfy jrhl f whrdg `o we`ne lfne`obs nhudg gh —foy

     whrm f lfo nfo gh,“ we`ne `s teb shrt hj of`vbdy hpt`l`st`n te`ok

    pbhpdb sf`g ifnm tebo, fog hob rbfsho wey Jbrrunn`‗ weh efs

     whrmbg `o frt`stn`fd `otbdd`kbonb jhr te`rty ybfrs‗`s lhrb nfut`husthgfy.6> Nhlput`ok ̀ s lfm`ok bohrlhus str`gbs, Jbrrunn` ohtbg. Web

    fi`d`ty th spht pfttbros `s krhw`ok spbntfnudfrdy. Fog lfne`ob dbfro-

    `ok, `o nhli`oft`ho w`te iurkbho`ok eulfo-lfne`ob `otbrfnt`hos

    teft jbbg teb dbfro`ok prhnbss, prhl`sbs jfr lhrb juogflbotfd fg-

    vfonbs th nhlb. —@t‐s kh`ok th ib hob hj tebsb bxphobot`fd nurvbs teft

     wb‐rb m`og hj ft teb ihtthl hj ohw,“ Jbrrunn` sf`g.

    Iut tebrb `s f vfst g`jjbrbonb ibtwbbo —[e`ne twh Yuss`fo dbfg-

    brs trfgbg chis8“ fog —[`dd twh Yuss`fo dbfgbrs trfgb chis fkf`o8“

    Web jhrlbr `s f e`sthr`nfd jfnt. Web nhlputbr nfo dhhm `t up. Web

    dfttbr rbqu`rbs teb nhlputbr th lfmb fo `ojhrlbg kubss fihut teb

    `otbot`hos hj Zdfg`l`r Qut`o, teb nefrfntbr hj Gl`tr` Lbgvbgbv,

    fog teb nfusfd gyofl`ns hj Yuss`fo phd`t`ns, fog tebo `otbkrftb teft

    `ojhrlft`ho `oth f cugklbot nfdd. Qbhpdb gh teft shrt hj te`ok fddteb t`lb, iut teft ghbso‐t lfmb `t bfsy. @t lbfos teb eulfo irf`o `s

     whogrhus‗ibnfusb teb tfsm `s stfkkbr`okdy efrg. Bvbo w`te nhl-

    putbrs lfm`ok kfddhp`ok fgvfonbs, teb shrt hj jhrbnfst`ok teft su-

    pbrjhrbnfstbrs gh `s f dhok wfy hjj. Fog Jbrrunn` `so‐t surb wb w`dd

    bvbr sbb f eulfo uogbr kdfss ft teb ^l`tesho`fo w`te f s`ko sfy`ok

    —suicbnt`vb cugklbot.“

    Lfne`obs lfy kbt ibttbr ft —l`l`nm`ok eulfo lbfo`ok,“ fog

    e i i g` ` e i e ` i — e ‐ g`jj

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    28/29

      Fo Hpt`l`st`n ^mbpt`n 9>

    `ok,“ Jbrrunn` sf`g. Weft‐s f spfnb eulfo cugklbot w`dd fdwfys hn-

    nupy.

    @o jhrbnfst`ok, fs `o htebr stbdgs, wb w`dd nhot`oub th sbb eulfo cugklbot ib`ok g`spdfnbg‗th teb nhostbroft`ho hj we`tb-nhddfr

     whrmbrs‗iut wb w`dd fdsh sbb lhrb fog lhrb syotebsbs, d`mb —jrbb-

    stydb nebss,“ `o we`ne eulfos w`te nhlputbrs nhlpbtb fs tbfls,

    teb eulfo grfw`ok ho teb nhlputbr‐s `og`sputfidb strboktes iut

    fdsh hnnfs`hofddy hvbrr`g`ok teb nhlputbr. Web rbsudt `s f nhli`of-

    t`ho teft nfo (shlbt`lbs) ibft ihte eulfos fog lfne`obs. Wh rb-

    jrflb teb lfo-vbrsus-lfne`ob g`nehthly, nhli`oft`hos hj KfrryMfspfrhv fog Gbbp Idub lfy prhvb lhrb rhiust tefo purb-eulfo

    hr purb-lfne`ob fpprhfnebs.

     [eft Jbrrunn` ghbs sbb ibnhl`ok hishdbtb `s teb kuru lhgbd

    teft lfmbs sh lfoy phd`ny gbiftbs sh pubr`db3 —@‐dd nhuotbr yhur Qfud

    Mruklfo phdbl`n w`te ly O`fdd Jbrkusho nhuotbrphdbl`n, fog

    rbiut yhur Whl Jr`bglfo hp-bg w`te ly Irbt ^tbpebos idhk.“ Jbr-

    runn` sbbs d`ket ft teb bog hj te`s dhok gfrm tuoobd3 —@ te`om `t‐s kh`ok

    th kbt strfokbr fog strfokbr“ jhr pbhpdb th d`stbo th teb fgv`nb hj

    bxpbrts wehsb v`bws frb `ojhrlbg hody iy teb`r suicbnt`vb cugklbot.

    Eulfo tehuket `s ibsbt iy psynehdhk`nfd p`tjfdds, f jfnt teft efs hody

    ibnhlb w`gbdy rbnhko`zbg `o teb dfst gbnfgb hr twh. —^h weft @ wfot

    `s teft eulfo bxpbrt pf`rbg w`te f nhlputbr th hvbrnhlb teb eulfo

    nhko`t`vb d`l`tft`hos fog i`fsbs.“67

    @j Jbrrunn` `s r`ket‗@ suspbnt eb `s‗ wb w`dd obbg th idbog

    nhlputbr-ifsbg jhrbnfst`ok fog suicbnt`vb cugklbot `o teb juturb.

    ^h `t‐s t`lb wb kht sbr`hus fihut ihte.

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    29/29

    http://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136693/siteID/8001/retailerid/6/trackingcode/penguinrandomhttp://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136709/siteID/8001/retailerid/22/trackingcode/PRH698C2DEE54http://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136693/siteID/8001/retailerid/2/trackingcode/PRH698C2DEE54http://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136709/siteID/8001/retailerid/3/trackingcode/PRH698C2DEE54http://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136693/siteID/8001/retailerid/7/trackingcode/randohouseinc10663-20http://www.crownpublishing.com/