43
Rummaging in the Government’s Attic Lessons Learned From More Than 1,000 FOIA Requests Michael Ravnitzky Phil Lapsley The Next HOPE, 2010

Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Embed Size (px)

Citation preview

Page 1: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

RummagingintheGovernment’sAtticLessonsLearnedFromMoreThan1,000FOIARequests

MichaelRavnitzkyPhilLapsley

TheNextHOPE,2010

Page 2: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Imagine

Page 3: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Imagine a gigantic database.

A database of documents.

Every document the U.S. Government has ever created.

Page 4: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

That database exists.

It even has a name.

Page 5: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

It’s called …

“Every document the U.S. Government has ever created and hasn’t gotten around to throwing 

out yet.”

Page 6: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Youcanquerythisdatabasebyusinganobscuresearchenginecalled“FOIA”

Page 7: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

FOIAistheFreedomofInformationAct

Electronic

Web

Queries are instant

Search engine is a computerand an algorithm

Tries to be helpful

Paper (mostly)

Postal service (mostly)

Queries take months, years

Search engine is a teamof humans

Tries to be helpful (mostly) …but sometimes comes acrossas petulant, even hostile

Page 8: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Everydocumentinthisdatabasehastwothingsincommon:

• Everydocumentinthedatabasewascreatedwithtaxpayermoney

• Everydocumentinthedatabaseshedslightonwhatyourgovernmentisupto

Page 9: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Butdocumentsinthedatabasestaythereunlessyouaskforthem

Nomechanismyetexiststosystematicallysharetheserecordswiththepublic.

RequestingdocumentsunderFOIAisasociallyuseful,necessary,andperfectlylegalformof“hacking.”

Example:recordsabouttheBPblowout

Page 10: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Whatwe’regoingtolearntoday

• What’saFOIArequest• Howtosubmitone• ToolsandtechniquesforFOIAhacking

We’lldothisbystudyingsomeexamplesfromthewebsiteGovernmentAttic.org,whoseproprietorshavefiledmorethan1,000FOIArequestsinthelastfewyears

Page 11: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

AFOIArequestcanbeassimpleasafour‐sentenceletter,emailorfaxd00ds,

OMG, this is totally a request under theFreedom of Information Act.

I request that you provide me with acopy of records regarding <whatever>.

I am an individual requesting thisinformation for noncommercial purposes.

I am willing to pay up to $XX for thisrequest if necessary.

Muchas gracias, Compadres!

J. Random Warez

Muststateit’saFOIArequest

Mustdescriberecordsyouwant

Muststatewillingnesstopayfees

Mustgiveinfofor“feecategory”

(Includeyourreturnaddressandcontactinfo!)

Page 12: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

TheFOIAprocess

Mail,fax,oremailrequest

Receiverequest

Logit

Sendackletter

(queue)

Searchfordocs

(queue)

(Declassify?)

Redact

Senddocs

Getackletter

(Get“norecords”letter)

Getgoodies

Norecords?

Weeks

Weeks/m

onths

Mon

ths/years

Reviserequestand/orappeal

You Da Gubmint

(Really?)

Page 13: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Fees:nutsandbolts• Agenciescanchargecertainreasonablesearchandduplicationfees

• Feesbasedonfour“feecategories”

• Mostrequestsendupbeingfreeorafewtensofdollars– …butbigfiles&searchescangetexpensive

• Limityourexposure–stateupfronthowmuchyou’rewillingtopay

• Iffeesarehigh,gettheFOIAofficertoexplainthemtoyouoverthephone

Page 14: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Fees:atollboothusedbysomeAgenciestodeterrequesters

• Feescanbeexpensiveforbigrequests

• SomeAgencieschargehighorerroneousfees

• Knowingthefourfeecategories,beingwillingtoreframerequests,discussingyourrequestwithFOIAstaffandtheirITpeoplecanhelp

• This“Tollbooththeory”explainsmanyoddFOIAsituations

Page 15: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Exemptions

• Agenciesareallowedtoblackout(“redact”)somestuffunderabunchofFOIAexemptions

• Exemptionsrangefromnationalsecuritytoprivacyconcernsto“goaway,that’sinternalstuff”

Page 16: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

RedactionslooklikeSwisscheesesprinkledwithgibberish

Swisscheese

Gibberish(Exemptioncodeslikeb1,b2,b7c,…)

(Holeswhereyourdocumenttextshouldbe)

Page 17: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Exemptionsareoftenmisused

• Someexemptionsarequestionable.Theirusedoesn’treallyholdupunderknowledgeablescrutiny.Beskeptical.

• WhenitcomestoFOIA,blesstheirhearts,Agenciesinadvertentlymakeerrors.Alot.

Page 18: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Thethreeexemptionsmostmisused

• Exemptionb(1)–currentlyandproperlyclassifiednationalsecurityinformation

• Exemptionb(2)–internalmaterials

• Exemptionb(5)–legallyprivilegedmaterial;usuallythe“deliberativeprocessprivilege”

Beespeciallyskepticalofthese.

Page 19: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Whatcanyoudo?

• Sendanappealletter– Google“foiaappeal”formoreinfo

• Thedeniallettershouldtellyouwheretosendyourappeal

• AppealsareoftensuccessfulifyouexplainwheretheAgencywentwrong

Page 20: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Goodnews:presumptionofdisclosure

PresidentBushandAttorneyGeneralAshcroftorderedagenciesto:– useFOIAexemptionsaggressivelytowithholdmaterial

PresidentObamaandAttorneyGeneralHolderorderedagenciesto:–releaseasmuchaspossible

–presumereleasewhenindoubt

–releaseevenexemptmaterialifthereis“noforeseeableharm”

2000 - 2008 2009 -

Page 21: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

FiveexamplesfromGovernmentAttic.Org

• Non‐commercialwebsitefoundedin2007

• HasfilednumerousFOIArequests

• Asapublicservice,postsdocumentsthataren’tavailableanywhereelse

• Shypeoplebutagreedtobeinterviewed

Page 22: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

FBIConfidentialFileRoomDossier

• CollectionofmemosidentifyingFBIfilessosensitivethattheyhadtobestoredinaspecialfileroom

• 1950–1972

• ServesasaroadmaptohistoricallysignificantFBIfiles

• Showsthetensionbetweentheneedtoisolatecertainfilesbutstillbeabletousethem–thisproblemstillexistsintheelectronicage

• Documentsthatdescribeinterestingdocuments

– Permitsrequestsforparticularfilesofinterest–byfilenumber!

Page 23: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

FBIConfidentialFileRoomDossierThe file on “Smuggling of AtomicBombs or Parts thereof and OtherWeapons of Mass Destructioninto the United States,” containshighly classified information …regarding United Statesdevelopments in the field ofatomic weapons and estimates ofSoviet developments in the fieldof atomic weapons, together withsuggested countermeasures.”

-- August 1, 1952

(And you thought the term “Weaponof Mass Destruction” was somethingPresident Bush came up with!)

Page 24: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

TheFBI’sHighVisibilityMemos

• ProducedwhenevertheFBIhandledaFOIArequestaboutanotableperson

• Thememospointtoanumberofnotablefiles

• GovernmentAtticwebmeistersaskedforthemafterseeingmentionof“highvisfiles”inanotherdocument

• Anotherexampleofdocumentsthatdescribeinterestingdocuments

Page 25: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

TheFBI’sHighVisibilityMemos

John Winston Ono Lennon,Deceased Former Member of theBeatles Musical Group:

“The bulk of the documents to bereleased are from a Security Matter –New Left investigatory file. Theinvestigation was initiated uponreceipt of information that Lennonmight engage in activities to disruptthe 1972 Republican NationalConvention …”

-- May 15, 1980

Page 26: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

DoDResaleActivitiesBoardofReview

• EstablishedbyCongresstoreviewwhichsexuallyexplicitmagazinesandvideoscouldbesoldorrentedonmilitarybases

• Anexampleofwheregovernmentandfreespeechcollide

• TheBoard’smeetingminutesandagendasanddecisionswererequested

Page 27: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

It’sadirtyjob,butsomebody’sgottodoit…

-- DoD Resale Activities Review Board, 1998, p. 80

Page 28: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Andtheverdictwas…

…andabout160others …andabout10others

Verboten! OK!

Page 29: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

USAFEntertainmentLiaisonOfficeWeeklyReports

• Army/Navy/AirForce/Marines/CoastGuardallhaveHollywoodoffices(“ELOs”)

• “Theremustbesomesortofperiodicreportingtoshowwhattheseguysweredoing”

• Gotalargenumberofweeklyreports

• AwindowintotheinfluenceDoDhasinHollywood

• Showshowthemilitarybuildsitsbrandthoughtheuseofmoviesandtelevision

Page 30: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

USAFEntertainmentLiaisonOfficeWeeklyReports

“Gumball 3000 – Edwards AFBPA referred a producer whoseproject involves music andentertainment stars going on a“Cannonball-like” race from LasVegas to Los Angeles. Theyasked if they could “race” onEdwards runways. According tothe producer the event would beon MTV and made into a feature.SAF/PAYE declined support asnot enough AF involvement towarrant it at this time, plus therace plans to end at a Playboymansion.”

-- October 11, 2005

Page 31: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

FEMA’sRAMPDatabase

• TheFederalEmergencyManagementAgencykeepsasetoflessonslearned

• RAMP:RemedialActionManagementProgram

• Showswhatworkedandwhatfailed

• EverythingfromSpaceShuttleChallengeraccidenttofloods/disasters/hurricanes/tornados

• GAgotaprintoutofthedatabaseafteraskingforthelessonslearnedonaflood,seeingamentionofRAMP,andaskingfortheRAMPdatabase

Page 32: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

FEMA’sRAMPDatabase

Oneofthousandsofsuggestionsforoperationalimprovement:

•“Insupportofrecenthurricanes,CR’sprimaryfunctionistohandoutflyerstovictims.IfIwereadisastervictimwithoutphone,electricityorsomemeanstogettoadisastercenteretc.andreceivedaflyerinformingmeofthisinformation,Iwouldbeinfuriated.”(Item1604‐I‐12,4/5/2006)

Page 33: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

SomeexamplesofitemstheGAdudesrequested

• FOIACaseLogs–listsofFOIArequests

• ContentsofAgencyINTRAnetsites

• ListofInspectorGeneralclosedinvestigations

• Listofagencyvideotapes

• List/CopiesofTSAEmergencyAmendments

• Listofoldeststill‐classifieddocumentsatanagency

• Listofrestrictedopinions(GAO)

• ReportsofanAgencytoCongressnotpostedonline

Documentspointingto

otherdocuments

Agencyself‐identifiedinterestingdocuments

Page 34: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

MorethingstheyFOIAed...

• ContentsofFOIATrackingFolders

• InternalAgencynewsletters

• CorrespondencewithMembersofCongress

• ArticlesfromNSATechnicalJournals

• NORAD9/11AirDefenserecordings

• AgencyMemorandaofUnderstanding(MOU/MOA)

• Listsofmovies/booksontheSpaceStation

• ComplaintstotheFCCaboutTVshowsPopularculture

“Peekbehindthescenes”

Page 35: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

DatabasesarefairgameunderFOIA

• FOIAappliestoelectronicrecordsanddatabases

• Askingfordigitaldatareducescopying/searchcosts

• AlsomakesiteasiertoshareontheInternet

• DiscussingselectionofdatabasefieldswithagencyITfolkscanhelpovercomeagencyobjections

Page 36: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

FOIA+database+web=coolwebsite

Page 37: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Problem:where’stheinterestingstuff?

• It’stoughforanindividualtoknowwheretheinterestingstuffis…– Orhowtoaskforit,evenifyouknowit’sthere

• Agencieshavenoincentivetotellarequesterwheretheinterestingstuffis– Orwhatmagicwordstousetoretrieveit

Page 38: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Solution:getthemtotellyou

• TheGAmuchachoshaveadoptedthegeneraltechniqueofsubmittingqueriesthatspurtheagencyitselftosignalwhereitsowninterestingstuffis

Examples:

• Identifyingrestrictedorunpublishedreports

• Lookingfor“missing”reportorfilenumbers

• Lookingatwhathistoricalrecordsarestillclassified

• Focusingonmaterialwithheldbytheagency

• Seeking“documentsthatidentifyotherdocuments”

Page 39: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Placesyoucangoformoreinformation

• nsarchive.org

• GetMyFBIfile.com

• GetGrandpasFBIfile.com

• OpenTheGovernment.org

• FOI‐Llistservarchives

• GovernmentAttic.org

Page 40: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Apartingchallenge

• Take30secondstothinkaboutsomethingyou’recuriousabout–arecentnewsstory,orwhatever

• Figureoutwhichgovernmentagencymighthavesomethingtodowithit

• Guesswhatlikelydocumentstheymighthave

• Requestthem!Today!

• Thankyounotesarealwaysagoodidea

• PostyourexperiencesontheHOPEForum!

Page 41: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

Appendix

Page 42: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

PrioritiesofaTypicalFOIAOffice

• Minimizeadministrativehassle• RebuffPITArequests• Minimizeagencyembarrassment/hassle• Notgiveupagency“property”(webuiltit–whyshouldtheygetitforfree?)

• Notreleasesensitivedocuments• Protectthepublictreasury• Whittleawaythebacklogofrequests

Page 43: Rummaging in the Government’s Attic: Lessons Learned From More Than 1,000 FOIA Requests

PrioritiesofaTypicalFOIARequester

• Obtaininteresting/usefulrecords(highrelevancy)

• Atminimumcostorfees(lowsystemprocessing)

• Andgetthemreasonablyquickly(lowlatency)