Upload
nasapmc
View
13.025
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Citation preview
1
Managing Revolutionary ResearchMarc G. Millis
2009-Feb-24
Management lessons from NASA's Breakthrough Propulsion Physics Project are presented. This project investigated such notions as gravity controland faster-than-light travel, assessing 10approaches, producing 16 journal articles, an award-winning website, and garnering positive media coverage for NASA – all for a total cost of $1.6 M spread over 1996 to 2002. The key tactic was to combine vision with rigor and to apply the lessons from history regarding scientific andtechnological revolutions.
Management lessons from NASA's Breakthrough Propulsion Physics Project are presented. This project investigated such notions as gravity controland faster-than-light travel, assessing 10approaches, producing 16 journal articles, an award-winning website, and garnering positive media coverage for NASA – all for a total cost of $1.6 M spread over 1996 to 2002. The key tactic was to combine vision with rigor and to apply the lessons from history regarding scientific andtechnological revolutions.
2
Revolutionary ResearchPioneering into Unknown
Evolutionary ResearchMastery of Known
Definition
3
To exceed the limits of prior technologymust seek entirely different technology
(Foster, Innovation - The Attacker’s Advantage, 1986)
Revolutionary ResearchWhen Needed?
Evolutionary
Resources
Performance Revolutionary
Revolutionary
Point of Diminishing Returns
4
Pioneers Masters
Revolutions, then Evolutions
After which thevalue is obvious to general community
Sophisticated
Reputable
Established
Unrefined
Not Yet Understood"Edgy"
Break from prior to search for new
Break from prior to search for new "Breakthrough" Point"Breakthrough" Point
5
Contrasting Attributes
Pioneers• Highest-Gain benefits sought• Expose ignorance - knowledge
gaps• Create new knowledge• Can NOT quantify comparisons• Intuitive progress
Masters• Maintain the knowledge base• Refine established knowledge• Quantify comparisons, “trades”• Procedural progress• Minimum Risk sought
Institutions• Must sustain preeminence & reputation• Must assess risk -vs- benefits• Must stay within budgets• Must produce progress
6
Revolutionary Research Challenges
• Breaks from the Norm ("Out of the Box")– Draws attention to what we don't know rather than
flexing our prowess for the known– Different type work: pioneering rather than mastery– Difficult to assess, quantify benefits
• Risky– Most ideas will fail to perform– Evokes hype, sensationalism, fringe– Evokes pedantic disdain– Difficult to sort viable 'crazy' ideas from the fringe– Success will be disruptive
7
Revolutionary ResearchManagement Challenges
Long-Term Goals Near-Term Progress
Visionary Credible
Divergent Options Need to Focus
Sufficient Investment Affordable
Unfamiliar Understandable
8
History Lessons• Organizational Patterns
Foster, Shepherd, Henderson, Utterback, Miller…
• Scientific RevolutionsKuhn (Paradigms), Dyson (Tool-Driven),
Clarke (3 Laws), Anderson (Horizon), Hamming (Great vs Good Researchers) Emme (Sci-fi/Sci-fact)
• Fringe TaintingLangmuir (Pathological Sci), Park (Voodoo
Sci), Sagan (Baloney filters), Baez (Crackpot Index), Kruger (Unskilled & Unaware)…
CompiledAdvice
9
Organizational Patterns
Resources
Performance
RevolutionaryRevolutionary
Point of Diminishing Returns
Mature = Entrenched
10
Vision Limiting Values • Prior values tailored to Method, not Goals
– Sail Effectiveness for Steam Ships ?– Specific Impulse for Warp Drive ?
• Values tailored to legacy customers, not future– Cold War Prowess -vs- Space Tourism– 1950's Colliers Magazine -vs-
• Affordability• Robotics• Environmental monitoring• Asteroid deflection• International collaboration
• Revolutionary ideas tend to take root outside the incumbent organizations
11
Sir Clarke's 3 LawsSir Arthur C. Clarke, (1972), Profiles of the Future, Bantam
1. When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.
2. The only way of discovering the limits of the possible is to venture a little way past them into the impossible.
3. Any sufficiently advanced technology is indistinguishable from magic.
12
Reflexive Dismissals“Space travel is utter bilge.”
- Dr. Richard van der Riet Wooley (one year before Sputnik 1)
“The secrets of flight will not be mastered within our lifetime, not within a thousand years.”
- Wilbur Wright (two years before Kittyhawk)
“Heavier than air flying machines are impossible,X-rays are a hoax.”
- William Thomson (Lord Kelvin)
But don’t forgetThere were also crazy ideas at the time that were crazy.
13
Reflections on Prior Scientific Revolutions
• COBE & WMAP• Hubble Space Telescope• Superconductivity• Internet networking• Nano Engineering
• Telescope → Galileo• X-ray diffraction → Watson/Crick
Dyson1997 Imagined Worlds
"Tool-Driven"
• Dark Matter• Dark Energy• Quantum Zero Point• Cosmological Const• GR/QM incompatibility• Gravity itself
• Epicycles → Heliocentric (Copernicus)• Gravity & Motion → Newton's laws• Michelson-Morley• Blackbody spectrum• Photoelectric effect• Wave/Particle dual
Kuhn1962 Structure of Scientific Revolutions
"Paradigm Shift"
Relativity(Einstein)
Quantum
14
• Have courage to tackle Important Problems– Grand challenges that will make a real difference, not just "safe" research
– Attackable; there is a way to begin solving the problem
• Start with independent thoughts and then collaborate
• Make steady progress, driven and focused
• Learn things beyond own work; "Knowledge is like compound interest"
• Redirect what is difficult to something easier (convert liabilities to assets)
• Honest with personal flaws & work to overcome
• Tolerate ambiguity
– Believe in self enough to proceed– Doubt self enough to honestly see flaws
Great Researchers & Important Problems1986 lecture, Richard Hamming, distinctions between good and great researchers
15
Via Science Fiction ?Emme, E., ed, (1982) Science Fiction & Space Futures Past & Present, American Astronautical Society History Series
• Science Fiction inspires pioneers - definitely !
• Science Fiction is not an accurate predictor
• Science Fiction akin to is brainstorming
Applying Clarke's 2nd Law:The only way of discovering the limits of the possible is to venture a little way past them into the impossible.
16
Via "Horizon Mission Methodology"Anderson, J. L, , (1996) "Leaps of the Imagination: Interstellar Flight and the Horizon Mission Methodology," JBIS, 49
1. Set Impossible Goals (break from mere extrapolations)
2. Assume it can be done in far future (Sci-fi brainstorming)
3. Look back from future, identifying critical assumptions
4. Identify the critical challenges (contrast knowledge to goals)
5. Identify knowledge gaps ("important questions")
Applying Clarke's 2nd Law:The only way of discovering the limits of the possible is to venture a little way past them into the impossible.
17
Next Challenge: Fringe Tainting
• 30% of project's unsolicited correspondence from amateurs sharing their ideas
• 30% of the amateur correspondence displayed delusion of grandeur and/or paranoia (Fringe)
• Voodoo Science [Park 2000]
• "Among the Fringe," S.Weinberger (2006 June 14), <http://www.defensetech.org/archives/002493.html>
• "Unskilled and Unaware…" [Kruger 1999] Personality and Social Psychology, V.77, p.121-
18
Responding to Amateur / Fringe
• Do not reply to correspondence displaying delusions of grandeur and/or paranoia (Fringe)
• When replying to those with nonviable ideas, use statements like, "appears to violate well-established laws of nature," rather than, "violates physics"
• Give them a next-step task to help them better understand their concept, rather than trying to teach them the principles…
• Make completion of that task a condition for continued correspondence, "to convince us… need to perform…to a higher standard of proof…"
19
Compilation of Advice from History• Identify diminishing returns, specifically where
revolutionary research is needed• Look across multiple disciplines and new 'tools' for
pioneers and possibilities (seeds of next S-curves)
• Employ the "Horizon" method to identify "Important Problems" - converting objections into objectives
• Develop more fundamental selection metrics than those used for prior methods (e.g. energy, not specific impulse)
• Familiarize the decision-makers with scholarly examples of emerging possibilities and how they apply to goals
• Dissect research approaches into short-term next-step research tasks (more affordable, less threatening)
• Promise to produce progress, not breakthroughs
20
Typical Reactionary Approach• New 'hot topic' gains attention.• Funds sought for hot topic only.• Other approaches not comparatively assessed
in a rigorous manner.
Typical Results• Success is defined in terms of whether the
approach worked.• Negative results not published.• In the event of a null result, support ebbs.• Window closes on all other approaches for
addressing these same challenges.
Typical Reactionary Approach• New 'hot topic' gains attention.• Funds sought for hot topic only.• Other approaches not comparatively assessed
in a rigorous manner.
Typical Results• Success is defined in terms of whether the
approach worked.• Negative results not published.• In the event of a null result, support ebbs.• Window closes on all other approaches for
addressing these same challenges.
Reactionary versus Strategic
Strategic Approach• Lead person acts as an impartial broker.• Various relevant approaches solicited and
comparatively evaluated.• Selection criteria concurred with customers and
researchers.• Scope of each task set to the minimal effort
needed to resolve an immediate “go / no-go”question.
Results Sought• Success defined as gaining reliable knowledge
to guide next steps.• Results, pro or con, published to set foundation
for future decisions.• Opportunity open for sequels to the positive
results, and to redirections around null results.
Strategic Approach• Lead person acts as an impartial broker.• Various relevant approaches solicited and
comparatively evaluated.• Selection criteria concurred with customers and
researchers.• Scope of each task set to the minimal effort
needed to resolve an immediate “go / no-go”question.
Results Sought• Success defined as gaining reliable knowledge
to guide next steps.• Results, pro or con, published to set foundation
for future decisions.• Opportunity open for sequels to the positive
results, and to redirections around null results.
21
NASA Breakthrough Propulsion Physics Project
Exceed the fundamental limits of existing propulsion by further advancing physics to discover the breakthroughs that could revolutionize spaceflight and enable interstellar voyages.
ProgrammaticConduct visionary
research in a credible manner.
TechnicalTarget the greatestchallenges of deep-
spaceflight.
22
Project Operating Principles
• Reliable – Define success as gaining reliable knowledge rather than claiming breakthroughs (puts emphasis where needed)
• Immediate – Focus on immediate make-or-break issues, unknowns, or curious effects (just enough for “go/no-go”), not the whole thing
• Iterated – Gain knowledge via cycles of short-term, incremental tasks
• Diversified – Support multiple, divergent research (not just hot topics)
• Measured – Track applicability and progress from cycles of research
• Impartial – Research selected via competitive peer assessments, where reviewers judge reliability of results, not feasibility of concept
• Empirical – Emphasis on experiments over pure theory or studies
• Published – Results published, both pro and con
23
Other "Revolutionary" Efforts
500(space portion)
"Freedom to Fail"Rotate staff ≤ 6yr to avoid entrenchment
DefenseDARPA
3.4"Don't let your preoccupation with reality stifleyour imagination"
Solicitations onlyoutside NASA
Revolutionary space systems
NASA Inst. Adv. Concepts(NIAC)
0.2- Emphasis on reliability, not claims
- Publish null results
- "Impossible" goals- Physics, not Tech- External solicitations
and in-house work
Emerging physics for propulsion
This project
Funding ($M/yr)
Taking Risks Avoiding Incumbent Limitations
ScopeOrg
24
Project Activities / Products• 1996-2002: Small Research Tasks• 2003-2008: Published Findings
– 10 Approaches– 16 Journal Articles– Compiled Book
25
Book: Frontiers of Propulsion Science
• AIAA Progress in Aeronautics and Astronautics Series
• Editors:– Marc Millis (NASA GRC)
– Eric Davis (Inst. Adv. Studies, Austin TX)
• 18 Authors
• 22 Chapters– Gravity control– Faster-than-light– Energy conversion– Project Management
• Publication Date: 2009-Feb-2
26
Visionary Credible
• Define success as gaining reliable knowledge rather than on achieving a breakthrough
• Ask reviewers to judge rigor and credibility, not feasibility (easier to detect a lack of rigor than to assess feasibility)
• Edgy physics pursued in aerospace, aerospace goals into physics• Convert objections into objectives (e.g. conservation of momentum -
to seeking alternate sources of reaction mass)
• Seek credible, published risk-takers
• Proposals contingent to prior peer-reviewed publication
• Submit progress to peer-reviewed journals– Free peer reviews– More credible publication venue
Project Tactics
27
Distinguishing Crackpots from VisionariesFood for thought, or a crackpot filled with half-baked baloney?
• Check for self-criticality– Does the author realize the critical make/break issues?– Is a discriminating test suggested?
• Check for awareness– Must demonstrate understanding of existing approaches– Must show advantage over existing approaches– Check for legitimate citations of compared approaches
• Rigorous on data, playful with interpretations– New idea must be consistent with credible data, but can disagree with
existing interpretations of the data (Copernicus analogy)– Check for legitimate citations of data
• Check track record– Prior publications– Prior products
28
Other Crackpot Filters
• Carl Sagan Baloney Detection Kithttp://www.carlsagan.com/revamp/carlsagan/baloney.html
• John Baez Crackpot Indexhttp://math.ucr.edu/home/baez/crackpot.html
29
Unfamiliar Understandable
• Identify the grand challenges and important problems (Horizon Method)
• Publish, publish, publish
• Include 'executive level' explanations in technical publications
• Be accessible to the Media, teaching, not advocating
Project Tactics, continued
30
Long-Term Goals Near-Term Progress
• Break long range goals into immediate questions– Critical Unknowns– Make-Break Issues– Curious Effects
• Narrow scope of research– Near-term results (1-3yr task duration)– Only address most critical questions, not the whole system
• Less threatening to reflexive challenges• More affordable• More likely to reach completion
• Traceability Map to explain linkage of research to goals and credible foundations
Project Tactics, continued
31
Traceability MapResearch linked to goals and credible foundations
1. Zero Propellant
2. Faster-Than-Light
3. Onboard Energy
Vacuum Battery ?
Goals
GrandChallenges
Concepts& Devices
Curious Effects, Unknowns, & Issues
Space Drives
Warp Drives & Wormholes
Quantum Fluctuations
General Relativity
Conservation Laws
Quantum Mechanics
Cosmology
Anomalous Rotation Rates(“Dark Matter”)
Anomalous Red-shifts(“Dark Energy”)
mpvp = mrvr
Knowledge
FoundationalPhysics
Compare Emerging Knowledge to Goals
32
Tachyon hypothesis
QED -vs- SED
Conservation laws for modifying spacetime
Tajmar apparent frame-dragging
Reactive spacetime
Reactive quantum vacuum
Inertia via quantum vacuum
Indigenous reaction matter
Reactive Machian frame
Degradable quantum vacuum
Causality, retrocausal paradoxes, & the definition
of time
Distinguish causal nonlocality from superluminal or
retrocausal signaling
Heim space theory
Tests of Mach-Lorentz thrusters
Felber gravity-repulsion
Superconductors/ gravitomagnetics
Hathaway tests gravity shield
Yamishita tests
Lifter tests
inertia as SED vacuum drag
Definition of exotic matter & energy conditions
No-signal theorems
Simultaneous linear coherence and entanglement
Polchinski nonlinear EPR FTL phone
Energetic heavy-water reactions
Leplace-Beltrami nonlinear QM in curved
spacetime
4
5
3,5
7
8-9
13
3,11
3,11
3
3,4,12
18
19
14,15,16
16
16
16
16
13,18
Sono-fusion19
1514
16
Electromagnetic -gravitational coupling
3,4,5,7,8,9,11,13
3,5,22
3
4,13
Existence of negative energy
4,15
4,12,18
4
15
Existence of negative matter
3,4
MEMS / NEMS quantum Casimir experiments
Traceability Map from Book (FPS)M
ASS: Thrust w
ithout propellant or beamed energy
SPEED: faster-than-light
ENER
GY
"Important Problems"Rigorous Foundations Goal-Driven Visions
Disciplines & Tools Curiosities Hypotheses & Tests Issues & Unknowns Concepts & Devices Categories
22#'s = Chapter citations
Electromagnetic techniques
MEG device
Sonolum' energy harvesting
Potapov water swirl chamber
Mechanical techniques
Space drive: sails
Space drive: fields
Inertia modification
Negative matterpropulsion
Spacetime modification / gravity control
Quantum approaches to gravity control
Brute fast
Spacetime modification for faster-than-light
Quantum nonlocality for faster-than-light
information
Novel nuclear processes
Quantum vacuum energy conversion
Modify gravitationl or inertial scalars
Yamishita electrogravitics
Forward g-dipoleLevi-Civita effect
Gravitationalwave propulsion
Modify quantum vacuum
Negative energy
Pinto levitation
Heim-Lorentz force
Alzofon antigrav'
Podkletnov gravity shield
Woodward Mach-Lorentz thruster
Oscillators & gyro antigravity
Biefeld-Brown
Corum DxB & Brito "EMIM"
Maclay dynamic Casimir effect
Warp drives & wormholes
Retrocausal communication.
Nachamkin resonant spheres
Koch voltage fluctuation coils
Ground state suppression
Cyclic CasimirShoulders EV energy
tapping
3,12
3
3
3,11
3,4,12,13
3,4
12
10
6
8,9
4,5,7-12
6
3,4
4
14
15
16
18
1919
1818
18
18
18,20
20
20
15
4
4
4
4,5
4,15
44
3,5
7
3,11,13
Forward Casimir battery
16
18
Computational tools & conventions
Quantum theory
Kinematics
Gravitomagneticapproximations
General relativity
Observational cosmology
Electrodynamics
Special relativity
Thermodynamics
Nuclear and particle physics
Anomalous supernovae redshifts
("Dark Energy")
Mach's Principle
Cosmological constant
Superconductivity
electromagnetic momentum in media
Quantum vacuum fluctuations
Cerenkov radiation
Relativisticlifetimes
Quantum entanglement
Sono-luminescence
10
21
3,11
19
16
14
14
3,4,12,13,15,16,18
15,18
3,4
Cosmic microwave background
3
Anomalous intra-galactic gravity
("Dark Matter") 3,4
33
Divergent Options Need to Focus
• Prioritization criteria consistent with goals and an emphasis on the reliability of information
• Select a small suite of options, divergent approaches
• Progress measured relative to applied science advancements (not technology readiness scale)
• Scoring system easily zeroes-out fringe submissions
• While individual tasks are near-term, sustained progress gained from iterating cycles of short-term tasks, adaptingto the results
Project Tactics, continued
34
Research Selection Process (1999)
Selection Criteria concurred by key players (1996-97)
Reviewers do NOT judge feasibility, instead judge:– Project Relevance– Credibility (reliable results upon which to make future decisions.) – Resources
2 - Stage Review Process– Peers numerically grade proposals
Minimum of 4 reviews per proposal (for statistics).Multiplicative, mandatory criteria.
– Customer team reviews scores to select winners
NASA in-house work subject to same review process
35
Project's Research Evaluation FactorsRelevance
1: Gain – Magnitude of performance improvement, assuming the technology ultimately reaches fruition.
2: Empiricism – Tangible effects or just theory?
3: Readiness – The present maturity of the topic/concept under study.
4: Progress – Magnitude of progress to be achieved, as measured by the difference in the readiness now, and the anticipated readiness upon completion of the task.
Credibility5: Foundations – Based on credible references.
6: Contrasts – Compared to current credible competing work.
7: Tests – Leading toward a discriminating test.
8: Results – Probability that the task will result in a reliable foundation for future decisions.
Resources9: Triage – Will it be done anyway or is it unique to this Project?
10: Cost – Funding required (reciprocal scoring factor).
11: Time – Time required to complete task (reciprocal scoring factor).
36
Conjecture Speculation Science Technology Commerce
Measuring Applied Science Progress
Scientific Method Steps4 Hypothesis empirically tested3 Hypothesis proposed2 Data collected1 Problem formulatedØ Pre-science:
• Anomalous effect observed, or• Knowledge gap realized
Ø 1 2 3 4 Ø 1 2 3 4 Ø 1 2 3 4
General Science
Critical Issues
Desired Effects
Technology Readiness Levels8 Flight Qualified7 Prototype demo in relevant environment6 Prototype demo5 Breadboard test in relevant environment4 Breadboard lab test3 Proof of concept2 Application concept formulated1 Basic principles reported
37
Evaluation Equation• Multiplicative criteria (1 failed criteria fails whole proposal)
• Scholastic gradations (A through F) where possible
• Team-generated criteria
Where:– A, B, C represent criteria scores– a, b, c are weighting factors(where 1 is the maximum value, and lower priorities are fractions of 1)– NA, NB, NC are normalizing functions– Cmin is a preset value to prevent the parenthetical term from
equaling zero, thereby making criteria C non-mandatory.
38
Proposal Scores
Standard deviation used to flag disparate scores
39
Traceability Map of Resulting Research
40
Sufficient Investment Affordable
Project Tactics, continued
• Overall progress made from sustaining cycles of short-term tasks and using the findings to affect the next solicitation (can interweave for better continuity)
• Support diversified portfolio of approaches (avoid tendency to only support 'hot' topics)
Solicitation
Research
Assess Findings
Solicitation
Research
Assess Findings
Solicitation
Research
41
Metrics of Project Value
1. Number of visionary notions converted into research tasks
2. Number of incremental unknowns, issues, or curious effects resolved
3. Degree of progress per task using Applied Science Progress Scale
4. Number of findings published in peer-literature
5. Number of citations of published works
6. Number of students inspired (can only count those who send comments)
7. Number of spin-offs
8. Number of educational materials produced
42
Revolutionary Research RisksReliability and Performance of Information, not Hardware
• Threshold of attention is when device can be engineered• Disclose only enough for independent verification of key
principles, not device, not best demo• After independent verification, advertise improved
version whose performance is more pronounced than verification demo
(For commercial research)Competitive advantage weakened from premature disclosure
• Sustain active scouting for ongoing development inside and external to the organization
• Pursue visionary research beyond the known - beyond what other organization address
• Forge widespread collaborations
Leadership stature damaged by neglectingrelevant advancements
• Emphasize reliable advances in knowledge, rather than requiring breakthroughs (the journey, not the destination)
• Collaborate with academia and other institutions for peer reviews
Credibility damaged by non-rigorous reporting
MitigationsRisks
43
Project Lessons LearnedDID WELL• Addressed a diversified set of relevant research approaches• Produced and published genuine research progress• Improved the credibility of topic• Cost-effectively accomplished all of the above
IMPROVEMENT NEEDED • Review opportunities as they emerge• Identify key issues that need the most attention• Link viable sponsors to reliable researchers
FOR NEXT TIME• Equal emphasis on in-house research for coverage of unaddressed
issues, continuity, and sustaining competence• Explore non-traditional techniques to leverage best of academia,
industry, and government – and to inspire and equip future pioneers
44
Summary
1. Combine Vision & Credibility
2. Identify "Important Problems"
3. Use appropriate comparisons(e.g. energy, not specific impulse)
4. Small increments of progress (affordable, less provocative)
5. Emphasis on physical observables
6. Publish reliably, publish often
Consider thepossibilities
Research rigorously & impartially
Evaluate rigor and impartiality – not
feasibility
Marc G. Millis, 2009