Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
CAREER%Panel%%
%%
1%
The%NSF%Review%Process%(aka%“why%does%it%take%so%long?”)%
2%
Binning%
Legend&Decision%point%
%Standard%flow%
Assign%to%
Program%
Officer%
%
Create%
panel%
(if%
applicable)%
%
Invite%
panelists%
(if%
applicable)%
Moderate%
panel%
(if%
applicable)%
Receive%%
*ad%Hoc/
mail%
reviews%
(if%
applicable)%
Request%
*ad%hoc/
mail%
reviews%
(if%
applicable)%
Individual%
panel%%
debrief%
Write%
RecomF
mendaGon%
memo%
Write/%
edit%
abstract%
Upload%
budget%
Program%
Officers%
discuss%%
coFfunding%
Obtain%
revised%
budget%%
(if%
required)%
NO%Out%of%Scope%
Decline%
YES%
In%Scope%
Fund%
Compliance%
%%%%%Check%
AdministraGve%
&%
financial%%
review%
Program%
Officer%
signs%off%“PO%Sign%off”%
Program%Officer%prepares%%
recommendaGon%
%
Program%%
Officer%
recommend%“PO%Recommend”%
%
ConversaGo
ns%between%
POs%and%DD%
Program%%
Officer%
recommend%“PO%Recommend”%
%
AdministraGve%
review%
IRB%
approval%
needed/%
present?%
%Noncompulsory%flow%%
Discussion,%
clarificaGon%and%
budget%
negoGaGon%with%
proposing%PI,%if%
necessary%
QuesMons%Proposal%revisions%
Further%Discussions%
Management%
signoff%
(DDConcur)%
NoGfy%PI%
Possible%
revision%or%
return%
without%
review%
Proposal%
submiUed%
Write%
RecomF
mendaGon%
memo%
The Big Picture
3%
Binning%
Legend&Decision%point%
%Standard%flow%
Assign%to%
Program%
Officer%
%
Create%
panel%
(if%
applicable)%
%
Invite%
panelists%
(if%
applicable)%
Moderate%
panel%
(if%
applicable)%
Receive%%
*ad%Hoc/
mail%
reviews%
(if%
applicable)%
Request%
*ad%hoc/
mail%
reviews%
(if%
applicable)%
Individual%
panel%%
debrief%
Write%
RecomF
mendaGon%
memo%
Write/%
edit%
abstract%
Upload%
budget%
Program%
Officers%
discuss%%
coFfunding%
Obtain%
revised%
budget%%
(if%
required)%
NO%Out%of%Scope%
Decline%
YES%
In%Scope%
Fund%
Compliance%
%%%%%Check%
AdministraGve%
&%
financial%%
review%
Program%
Officer%
signs%off%“PO%Sign%off”%
Program%Officer%prepares%%
recommendaGon%
%
Program%%
Officer%
recommend%“PO%Recommend”%
%
ConversaGo
ns%between%
POs%and%DD%
Program%%
Officer%
recommend%“PO%Recommend”%
%
AdministraGve%
review%
IRB%
approval%
needed/%
present?%
%Noncompulsory%flow%%
Discussion,%
clarificaGon%and%
budget%
negoGaGon%with%
proposing%PI,%if%
necessary%
QuesMons%Proposal%revisions%
Further%Discussions%
Management%
signoff%
(DDConcur)%
NoGfy%PI%
Possible%
revision%or%
return%
without%
review%
Proposal%
submiUed%
Write%
RecomF
mendaGon%
mem%
Compliance%%
Check%Proposal%submiUed%
Possible%
revision%or%
return%without%
review%
Compliance+checks+look+for:+• Submi5ed+within+window+• Includes+all+required+items+
(e.g.,+dept+chair+le5er+for+CAREER)+
• Font,+margins,+page+count+meet+GPG+(increasingly+automated)+
• Project+summary+includes+defined+IM+&+BI+
• Budget+is+in+allowed+range+• No+coMPIs+(for+CAREER)+• No+unauthorized+
a5achments+(e.g.,+“support”+le5ers,+charts)+
• Bio+sketch+follows+format+• Isn’t+a+duplicate+• Etc.+
In+most+cases,+non+compliance+means+proposal+is+returned+without+review+4%
Binning%
Legend&Decision%point%
%Standard%flow%
Assign%to%
Program%
Officer%
%
Create%
panel%
(if%
applicable)%
%
Invite%
panelists%
(if%
applicable)%
Moderate%
panel%
(if%
applicable)%
Receive%%
*ad%Hoc/
mail%
reviews%
(if%
applicable)%
Request%
*ad%hoc/
mail%
reviews%
(if%
applicable)%
Individual%
panel%%
debrief%
Write%
review%
analysis%
Write/%
edit%
abstract%
Upload%
budget%
Program%
Officers%
discuss%%
coFfunding%
Obtain%
revised%
budget%%
(if%
required)%
NO%Out%of%Scope%
Decline%
YES%
In%Scope%
Fund%
Compliance%
%%%%%Check%
AdministraGve%
&%
financial%%
review%
Program%
Officer%
signs%off%“PO%Sign%off”%
Program%Officer%prepares%%
recommendaGon%
%
Program%%
Officer%
recommend%“PO%Recommend”%
%
ConversaGo
ns%between%
POs%and%DD%
Program%%
Officer%
recommend%“PO%Recommend”%
%
AdministraGve%
review%
IRB%
approval%
needed/%
present?%
%Noncompulsory%flow%%
Discussion,%
clarificaGon%and%
budget%
negoGaGon%with%
proposing%PI,%if%
necessary%
QuesMons%Proposal%revisions%
Further%Discussions%
Management%
signoff%
(DDConcur)%
NoGfy%PI%
Possible%
revision%or%
return%
without%
review%
Proposal%
submiUed%
Write%
RecomF
mendaGon%
memo%
5%
Assign%to%Program%
Officer%
%
Create%panel%
(if%applicable)%
%
Invite%panelists%
(if%applicable)%
Request%
“ad%hoc”/mail%
reviews%
(if%applicable)%
• Binning+groups+related+proposals+together,+and+includes+swapping+and+sharing+between+programs+
• Program+officer+assigned+based+on+research+area+• Most+proposals+reviewed+on+panels+–+it+takes+Ume+to+find+the+right+people!+• Niche+areas+someUmes+have+ad+hoc+reviewers+in+addiUon+(or+instead)+of+
panels+• Most+proposals+reviewed+on+one+panel,+but+someUmes+by+two+panels+within+
the+same+program+or+by+two+programs+
Binning%
Binning%
Legend&Decision%point%
%Standard%flow%
Assign%to%
Program%
Officer%
%
Create%
panel%
(if%
applicable)%
%
Invite%
panelists%
(if%
applicable)%
Moderate%
panel%
(if%
applicable)%
Receive%%
*ad%Hoc/
mail%
reviews%
(if%
applicable)%
Request%
*ad%hoc/
mail%
reviews%
(if%
applicable)%
Individual%
panel%%
debrief%
Write%
RecomF
mendaGon
memo%
Write/%
edit%
abstract%
Upload%
budget%
Program%
Officers%
discuss%%
coFfunding%
Obtain%
revised%
budget%%
(if%
required)%
NO%Out%of%Scope%
Decline%
YES%
In%Scope%
Fund%
Compliance%
%%%%%Check%
AdministraGve%
&%
financial%%
review%
Program%
Officer%
signs%off%“PO%Sign%off”%
Program%Officer%prepares%%
recommendaGon%
%
Program%%
Officer%
recommend%“PO%Recommend”%
%
ConversaGo
ns%between%
POs%and%DD%
Program%%
Officer%
recommend%“PO%Recommend”%
%
AdministraGve%
review%
IRB%
approval%
needed/%
present?%
%Noncompulsory%flow%%
Discussion,%
clarificaGon%and%
budget%
negoGaGon%with%
proposing%PI,%if%
necessary%
QuesMons%Proposal%revisions%
Further%Discussions%
Management%
signoff%
(DDConcur)%
NoGfy%PI%
Possible%
revision%or%
return%
without%
review%
Proposal%
submiUed%
Write%
RecomF
mendaGon%
memo%
6%
Program%Officer%
prepares%
recommendaGon%
Moderate%panel%
(if%applicable)%
• The+most+public+part+of+a+program+officer’s+job+is+moderaUng+a+panel+• AXer+the+panel,+recommendaUons+discussed/approved+by+management+• Panels+provide+advice+• Program+officers+provide+recommendaUons+• Management+provides+approval+• DGA+provides+commitment+
Binning%
Legend&Decision%point%
%Standard%flow%
Assign%to%
Program%
Officer%
%
Create%
panel%
(if%
applicable)%
%
Invite%
panelists%
(if%
applicable)%
Moderate%
panel%
(if%
applicable)%
Receive%%
*ad%Hoc/
mail%
reviews%
(if%
applicable)%
Request%
*ad%hoc/
mail%
reviews%
(if%
applicable)%
Individual%
panel%%
debrief%
Write%
RecomF
mendaGon
memo%
Write/%
edit%
abstract%
Upload%
budget%
Program%
Officers%
discuss%%
coFfunding%
Obtain%
revised%
budget%%
(if%
required)%
NO%Out%of%Scope%
Decline%
YES%
In%Scope%
Fund%
Compliance%
%%%%%Check%
AdministraGve%
&%
financial%%
review%
Program%
Officer%
signs%off%“PO%Sign%off”%
Program%Officer%prepares%%
recommendaGon%
%
Program%%
Officer%
recommend%“PO%Recommend”%
%
ConversaGo
ns%between%
POs%and%DD%
Program%%
Officer%
recommend%“PO%Recommend”%
%
AdministraGve%
review%
IRB%
approval%
needed/%
present?%
%Noncompulsory%flow%%
Discussion,%
clarificaGon%and%
budget%
negoGaGon%with%
proposing%PI,%if%
necessary%
QuesMons%Proposal%revisions%
Further%Discussions%
Management%
signoff%
(DDConcur)%
NoGfy%PI%
Possible%
revision%or%
return%
without%
review%
Proposal%
submiUed%
Write%
RecomF
mendaGon%
memo%
7%
Write%
RecomF
mendaGon%
memo%
Management%
signoff%
(DDConcur)%
• Program+officers+write+a+“recommendaUon+memo”+to+explain+to+management+why+a+proposal+is+being+declined+
Binning%
Legend&Decision%point%
%Standard%flow%
Assign%to%
Program%
Officer%
%
Create%
panel%
(if%
applicable)%
%
Invite%
panelists%
(if%
applicable)%
Moderate%
panel%
(if%
applicable)%
Receive%%
*ad%Hoc/
mail%
reviews%
(if%
applicable)%
Request%
*ad%hoc/
mail%
reviews%
(if%
applicable)%
Individual%
panel%%
debrief%
Write%
review%
analysis%
Write/%
edit%
abstract%
Upload%
budget%
Program%
Officers%
discuss%%
coFfunding%
Obtain%
revised%
budget%%
(if%
required)%
NO%Out%of%Scope%
Decline%
YES%
In%Scope%
Fund%
Compliance%
%%%%%Check%
AdministraGve%
&%
financial%%
review%
Program%
Officer%
signs%off%“PO%Sign%off”%
Program%Officer%prepares%%
recommendaGon%
%
Program%%
Officer%
recommend%“PO%Recommend”%
%
ConversaGo
ns%between%
POs%and%DD%
Program%%
Officer%
recommend%“PO%Recommend”%
%
AdministraGve%
review%
IRB%
approval%
needed/%
present?%
%Noncompulsory%flow%%
Discussion,%
clarificaGon%and%
budget%
negoGaGon%with%
proposing%PI,%if%
necessary%
QuesMons%Proposal%revisions%
Further%Discussions%
Management%
signoff%
(DDConcur)%
NoGfy%PI%
Possible%
revision%or%
return%
without%
review%
Proposal%
submiUed%
Write%
review%
analysis%
8%
Binning%
Legend&Decision%point%
%Standard%flow%
Assign%to%
Program%
Officer%
%
Create%
panel%
(if%
applicable)%
%
Invite%
panelists%
(if%
applicable)%
Moderate%
panel%
(if%
applicable)%
Receive%%
*ad%Hoc/
mail%
reviews%
(if%
applicable)%
Request%
*ad%hoc/
mail%
reviews%
(if%
applicable)%
Individual%
panel%%
debrief%
Write%
review%
analysis%
Write/%
edit%
abstract%
Upload%
budget%
Program%
Officers%
discuss%%
coFfunding%
Obtain%
revised%
budget%%
(if%
required)%
Decline%
In%Scope%
Fund%
Compliance%
%%%%%Check%
AdministraGve%
&%
financial%%
review%
Program%
Officer%
signs%off%“PO%Sign%off”%
Program%Officer%prepares%%
recommendaGon%
%
Program%%
Officer%
recommend%“PO%Recommend”%
%
ConversaGo
ns%between%
POs%and%DD%
Program%%
Officer%
recommend%“PO%Recommend”%
%
AdministraGve%
review%
IRB%
approval%
needed/%
present?%
%Noncompulsory%flow%%
Discussion,%
clarificaGon%and%
budget%
negoGaGon%with%
proposing%PI,%if%
necessary%
Further%Discussions%
Management%
signoff%
(DDConcur)%
NoGfy%PI%
Proposal%
submiUed%
Write%
review%
analysis%
AdministraGve%
&%
financial%%
review%
NO%Out%of%Scope%
YES%
QuesMons%Proposal%revisions%
Possible%
revision%or%
return%
without%
review%
• Once+decision+is+made+to+recommend+an+award,+Program+officer+contacts+the+PI,+and+may+ask+for:+• Revisions+to+budget+• ClarificaUons+to+scope+• Responses+to+quesUons+raised+by+the+panel+• RelaUonship+to+other+awards+• IRB+approval+if+human+subjects+involved+• DraX+of+an+abstract+for+public+site+
• Once+all+the+info+is+provided+and+reviewed,+program+officer+prepares+recommendaUon+memo+for+management+approval+
Write/%
edit%
abstract%
Obtain%
revised%
budget%%
(if%required)%
Program%
Officer%
signs%off%
IRB%
approval%
needed/%
present?%
NoGfy%PI%Write%
review%
analysis%
Mgmt%
signoff%
(DDConcur)%
9%
The%Clouds%Part%
10%
The%Steps%In%Review%• Compliance,%binning,%trading,%etc.%• IdenMfy%panelists%• Panelists%read%proposals%and%write%reviews%• The panel • Coordinate%with%other%panels%on%the%same%program%or%other%programs%coZreviewing,%get%management%preliminary%approval%
• Contact%PIs%and%ask%for%clarificaMons%• Write%recommendaMon%memo%and%get%management%approval%
• DGA%award%to%the%insMtuMon%• Timeline:%70%%of%proposals%awarded/declined%within%six%months%
11%
Panel%Overview%
• Assigning%proposals%to%a%program%
• Assigning%proposals%to%a%panel%• ConsMtuMng%a%panel%%• Panel%day%deliberaMons%• Panel%recommendaMons%%
%
12%
Before%the%Panel%
• Assigning%proposals%to%a%program%– Program%Directors%(PDs)%read%proposals%and%“swap”%acer%obtaining%
PI’s%permission%
• ConsMtuMng%a%panel%(typically%senior%members%of%research%community%with%appropriate%experMse%and%breadth%of%experience;%someMmes%recent%CAREER%awardees)%
• Assigning%proposals%to%panelists%–%about%a%month%before%panel%– Panelists%asked%about%COIs%– PD%assigns%or%Panelists%provide%a%spreadsheet%of%review%preferences%– Final%assignments%ZZ%minimum%of%3%reviews%per%proposal;%typically%7Z10%
proposals%per%panelist%– Reviews%requested%before%panel%day%
• Obtaining%ad%hoc%reviews%where%appropriate%
13%
Panel%Day%DeliberaMons%
• Program%Director%(PD)%presents%and%discusses%“panel%charge”%– Panelists%and%PD%leave%the%room%while%proposal%with%which%have%%COI%is%
discussed%– Discussion%of%review%criteriaZZIntellectual%Merit%(IM)%&%Broader%Impact%
(BI)%including%integraMon%of%research%and%educaMon,%as%well%as%Data%Management%Plan%&%Post%Doc%Mentoring%Plan%(if%post%docs%included%in%proposal)%
• Each%proposal%is%discussed%as%to%IM%and%BI%and%placed%into%a%recommendaMon%category%(typically%Highly%CompeMMve,%CompeMMve,%Low%CompeMMve,%and%Not%CompeMMve)%
• Panel%summary%is%wrilen%by%a%panelist%who%serves%as%“scribe”%and%approved%by%all%nonZconflicted%panelists%
• Panelists%may%change%their%reviews%to%reflect%issues%raised%during%the%panel%discussion%
• RecommendaMons%– Panel%makes%recommendaMons%not%funding%decisions%
%14%
What%do%Panels%Look%for?%Heilmeier's%Catechism%
• What%are%you%trying%to%do?%ArMculate%your%objecMves%using%absolutely%no%jargon.%%(Clear%summary)%
• How%is%it%done%today,%and%what%are%the%limits%of%current%pracMce?%(Related%work)%
• What's%new%in%your%approach%and%why%do%you%think%it%will%be%successful?%(Include%preliminary%work)%
• Who%cares?%If%you're%successful,%what%difference%will%it%make?%(Impact)%
• What%are%the%risks%and%the%payoffs?%%• How%much%will%it%cost?%How%long%will%it%take?%(Budget)%• What%are%the%midterm%and%final%"exams"%to%check%for%
success?%(EvaluaMon%plan)%%
– George%H.%Heilmeier,%President%and%CEO%of%Bellcore%15%
Top%Ten%Mistakes%to%Avoid%in%WriMng%CAREER%Proposals%
CISE%Mixed%Advisory%Taskforce%on%Technology,%
EducaMon,%Research%and%Science%(CISE%MATTERS)%
16%
Number%10:%Fonts%Too%Small%
• Small%fonts%promote%reader%faMgue%
• Reviewers%HATE%small%fonts%
• GPG%mandates:%• 11%point%font%minimum%
• 1%inch%margins%
• 6%lines%max%per%verMcal%inch%
17%
Number%9:%Figures%Illegible%
• Avoid%“crowded”%visuals%
• Don’t%assume%reader%will%print%in%color%
• Use%vector%graphic%formats%
18%
Number%8:%Acronyms%
• Acronyms%are%UGLY,%and%make%text%hard%to%read.%
• Write%in%ENGLISH,%not%KLINGON!%
Reliance%on%acronyms:%A%pile%of%Base%StaMon%(BS)%
19%
Number%7:%Dissing%the%CompeMMon%
• Good%idea:%CiMng%others’%work%• Bad%idea:%SlighMng%others’%work%
(“Others’%work”%might%be%sirng%on%the%panel)%20%
Number%6:%Poor%disMncMon%between%preliminary%results%and%proposed%work%
• Make%a%clear%demarcaMon%
• DisMnguish%your%results%from%others’%
• Provide%clear%road%map%for%future%work%
??
21%
Number%5:%Lackluster%EducaMon%Plan%
• Should%be%integrated%with%research%plan%• Think%beyond%your%present%teaching%duMes%
22%
Number%4:%“It%wasn’t%clear%…”%
• LongZwinded%explanaMons%• Too%many%superfluous%details%
• Poor%organizaMon%of%thoughts%into%words%
Symptoms:%
Remedies:%
• Use%fewer%words%• Read%first%two%pages%aloud%• “Make%every%word%tell”%
23%
Number%3:%Dull%Broader%Impacts%
• Broader%Impacts%ask:%
– How%will%this%work%change%society?%• Don’t%confuse%this%with%“extracurricular%acMviMes”%not%supported%by%the%research%plan%
• Outreach%plans%should%be%substanMated%
24%
Number%2:%%Confining%yourself%to%your%PhD%work%
• CAREER%proposal%should%be%forwardZlooking%• Move%above%and%beyond%your%PhD%work%
• “Imagine%a%world%…”%
(yes)% (no)%25%
Number%1:%%Research%Plan%lacking%Cohesion%
• Don’t%staple%together%unrelated%ideas%
• Don’t%offer%a%laundry%list%with%no%prioriMzaMon%
• Don’t%make%everything%look%like%a%nail%to%your%one%hammer%
• Tell%a%story%with%your%narraMve%
26%
QuesMons?%
27%
Image%alribuMons%
28%
p.%2:%%
hlp://mediag.com/news/aZpeekZbehindZtheZcurtain/%%
pp.%3ZZ9:%
%J.%Epstein,%NSF%
%p.%10%
%
hlp://www.wallpedes.com/p/11129%%
p.16%%
hlps://pinctadafucata.files.wordpress.com/2010/12/obamaZfingerZwagZcroppedZprotoZcustom_2.jpg%hlp://brentwhite.files.wordpress.com/2013/07/colbert.jpg%
hlp://bleacherreport.com/arMcles/1513431ZwatchZdikembeZmutomboZgetZhisZfingerZwagZonZinZhilariousZnewZgeicoZad%%
p.%17:%%
P.%Regalia,%NSF%
%p.%18:%
%hlp://file.scirp.org/Html/1_970047_422.htm%
hlp://www.mathworks.com/matlabcentral/fileexchange/screenshots/23744/original.jpg%%
p.%19:%%
P.%Regalia,%NSF%
%p.%20:%
%hlp://www.bleachernaMon.com/2011/08/27/brelZjacksonZprobablyZwontZbeZcalledZupZinZseptemberZandZotherZbullets/homerZangryZphone/%hlp://www.angrymetalguy.com/angryZmetalZguyZspeaksZonZthingsZmakeZbeingZangryZmetalZguyZhard/%hlp://www.hlburkeblog.com/2015/01/getZoverZitZwriters.html%
%p.%21:%
%hlp://mavismarMn.com/2014/05/18/nowZacMveZpartZlife/%
hlp://qiroadmap.org/howZtoZuseZtheZqiZroadmap/%
%
p.%22:%%%
hlp://blog.knetbooks.com/boringZcollegeZclasses/%hlp://www.dreamsMme.com/stockZimageZboringZclassZimage541051%
hlp://alboum.com/2011/03/ifZyouZthinkZdicMonariesZareZboring/%
%p.%23:%
%hlp://employeeZitZtraining.co.uk/2014/01/everyZlilleZhelps/%
%
"Vigorous%wriMng%is%concise.%A%sentence%should%contain%no%unnecessary%words,%a%paragraph%no%unnecessary%sentences,%for%the%same%reason%that%a%drawing%should%have%no%unnecessary%lines%and%a%machine%no%unnecessary%parts.%This%requires%not%that%the%writer%make%all%his%sentences%short,%or%that%he%avoid%all%detail%and%treat%his%subjects%only%in%outline,%but%that%every%word%tell."%%
Strunk%and%White,%The%Elements%of%Style,%ISBNZ13:%9780205309023%%
p.%24:%%
hlp://www.bmyersphoto.com/BWXRAY/minerals13.html%
hlp://en.wikipedia.org/wiki/AnMque_radio%hlp://simple.wikipedia.org/wiki/Integrated_circuit%
hlp://www.apple.com/%%
p.%25:%
%hlp://mycreaMvecatalyst.com/quantumZleapZyourZcash/%
hlp://www.cheapcarsforsaleunder1000.net/cheapZcarZforZsale/%%
p.%26:%
%hlp://www.thinkgeek.com/product/61b7/%
hlp://pathfindersocware.com/2009/03/whatZshouldZourZsocwareZdo/%hlp://imgkid.com/hammerZandZnailsZclipart.shtml%
%
p.%27:%%
hlp://www.theguardian.com/music/2010/sep/13/firstZklingonZoperaZlicsZoff%