View
217
Download
3
Category
Tags:
Preview:
Citation preview
Facilities Information Management System Data
Validation Update for FY15
U.S. Department of Energy
Argonne National Laboratory
Annual FIMS Comprehensive Training
May 7, 2015
The FIMS Validation Process
Overview of the Validation Process
Statistical standard for validating the accuracy of key FIMS data elements and performance measures against source data Identifies performance measures and data elements
for validation Establishes confidence level
The validation process is based on a model to achieve a 90% confidence level in the quality of the data
Validating the processes applied by a site to populate FIMS
Overview of the Validation Process (continued)
Validation Goal To infer a level of accuracy within the whole
database. Therefore, we can also infer the ability of the data to support accurate management decisions.
A key goal of the validation process is to standardize procedures throughout the Department regardless of who applies it
FY2015 FIMS Validation
For FY2015, 4 validations will be performed One validation will include a random sample of
DOE owned buildings, trailers, and Other Structures and Facilities (OSF)
The second validation will include all DOE leased buildings, trailers and Other Structures and Facilities (OSF) and GSA owned and leased buildings
The third validation will include DOE owned, DOE leased and withdrawn from public domain land
FY2015 FIMS Validation (continued)
For FY2015, 4 validations will be performed The fourth validation will include
Archived/Disposed assets with an Archive Date between October 1, 2012 and September 30, 2014
Contractor leased assets will not be validated Bridge Safety Inspections will be verified
FIMS Validation Process
Inbrief
Determine Sample Size
Generate Random Sample
Review Outliers and Generate Outlier Report
Generate DOE Leased and GSA
Report
Generate Land Report
Generate Archived/Disposed
Report
Generate Bridge Safety Inspection
ReportSource Documents
Data Element Review
Process Verification
Desktop Validation Walkthroughs
Prepare Scorecards
Prepare Outbrief & Follow-up Actions
Sample Set Size
DOE Owned Sample Size
Sample size will consist of a maximum of 25 DOE owned assets as indicated in the Sample Size chart
The sample size will be used as input to generate the FIMS DOE Owned Data Validation Reporting Form
Sample Size Chart - for a 90% Confidence Level
# of Assets* Sample Size
1 1
2 2
3 3
4 4
5-6 5
7 6
8-9 7
10-11 8
12-14 9
15-16 10
17-20 11
21-23 12
# of Assets* Sample Size
24-27 13
28-33 14
34-39 15
40-46 16
47-56 17
57-69 18
70-87 19
88-113 20
114-154 21
155-232 22
233-242 23
>242 25
* DOE owned Buildings, Trailers, and Other Structures and Facilities (OSF)
Sample Set – DOE Leased, GSA Owned and GSA Leased Assets DOE leased and GSA sample set to be
validated 100% of a site’s DOE leased buildings, trailers and
OSF assets
AND 100% of a site’s GSA owned and GSA leased buildings
assets
Sample Set – Land Assets
Land sample set DOE owned, DOE leased, and withdrawn from
public domain land records will be validated Sites with 25 or less land records will validate all
(100% of) their land records
OR Sites with more than 25 land records will validate 25
land records from a random pull of their land records
Sample Set – Archived/Disposed Assets Archived/Disposed sample set
Archived records with an Archive Date between October 1, 2012 and September 30, 2014 will be validated Sites with 25 or less records will validate all (100% of)
their archived records
OR Sites with more than 25 archived records will validate
25 records from a random pull of their archived records Assets archived as ‘Administrative Correction’ will
not be validated
FIMS: Extracting the Data and Generating the Data Validation Reporting Forms
Before extracting the FIMS data
Obtain the total number of DOE
owned buildings, trailers and
OSF assets at the site The FIMS Ad Hoc Report – Current query tool can be
used to determine the total number of assets Determine sample size using the Sample Size
chart
FIMS Standard Report for generating the Random Sample Set
To extract the random sample set for DOE owned buildings, trailers and OSF to be validated: Run the FIMS Report #069 – Random Asset
Generator (FRPC Data Validation) Report #069 will generate the DOE
Owned Data Validation Reporting Form
What are outliers? Additional assets that may manually be added
to the DOE owned random asset validation set Items to look for when selecting ‘Outlier’ assets:
High RPV but small footprint High Deferred Maintenance assets Large footprint but small RPV Very important assets Newly acquired assets Otherwise of interest
Use the FIMS Ad Hoc Report – Current query tool to assist the validation team in selecting outliers to include with the DOE owned random sample set Note the Property ID(s)
Selecting Outliers for DOE Owned Assets #070 FRPC Data Element Report
FIMS standard report #070 FRPC Data Element Report can be used to add ‘outliers’ to the validation random sample This report can be generated for specific Property IDs
and output to the MS Excel format Copy and paste the MS Excel output into the DOE
Owned Data Validation Reporting Form generated by FIMS standard report #069. Copy the data to the worksheet labelled FIMS below the existing data.
FIMS Standard Report for Validating DOE Leased and GSA assets To extract the DOE leased and GSA
sample set to be validated: Run the FIMS Report #069a – DOE Leased
and GSA Assets (FRPC Data Validation) Report #069a will generate the DOE
Leased and GSA Data Validation Reporting Form
FIMS Standard Report for Validating Land assets To extract the DOE owned, DOE leased
and withdrawn from public domain land sample set to be validated Run the FIMS Report #069b – Land (FRPC
Data Validation) Report #069b will generate the DOE Land
Data Validation Reporting Form
FIMS Standard Report for Validating Archived/Disposed assets To extract the Archived/Disposed sample
set to be validated Run the FIMS Report #069c –
Archived/Disposed (FRPC Data Validation) Report #069c will generate the Disposition
Data Validation Reporting Form
OSF Bridge Safety Inspection data extract
All sites with bridges will be required to verify the Safety Inspection information for all public access and controlled access train bridges and public access vehicular bridges at the site
Use report #068 Bridge Safety Inspection Report to extract the data from the FIMS database for the verification
Timeline for data extraction
The FIMS validation data extracts are to occur on the first day of the validation
Data Validation Reporting Forms
Site level operating cost fields are automatically populated with FIMS data on the DOE Owned Data Validation Reporting Form
Instructions for using these forms can be found on the FIMS website on the Data Validation page, http://fimsinfo.doe.gov/data_validation.htm
Source Documents
Source Documents
FIMS data element values must be validated against independent source documents
What are Source Documents?
Source Document – at each site, documents containing detailed comprehensive site data that should be the “accurate data of record” Can be a “hard” or “soft” document Information contained is authoritative, has a Subject
Matter Expert and is used in decision-making Document is dated
Source Documents for Archived/Disposed Assets Acceptable source documents include:
Site Subject Matter Expert in charge of asset disposition Real Estate Disposition Manager Accounting
Photos showing before and after of asset site Google Earth
Source Document Update Frequencies
Source Documents are expected to be updated annually for the following:
Usage Code Mission Dependency
Status Annual Operating Costs
Outgrant Indicator Annual Actual Maintenance
Excess Indicator Sustainability
Estimated Disposition Year Total No of Occupants
Using Organization Total No of Federal Employees
Asset % Utilized Total No of Contractor Employees
RPV Total Number of Other Personnel
Deferred Maintenance Meters
Source Document Update Frequencies (continued)
Source Documents are expected to be updated annually for the following:
Repair Needs GSA Notification Submitted (date)
Inspection Date GSA Notification Accepted (date)
Adjustment Cost Annual Rent
Determination Date for DOE Disposal
Source Document Update Frequencies (continued)
Source data for Bridge Safety Inspection is expected to be updated with the same frequency as the required safety inspection Controlled access and publicly accessible train
bridges – one inspection per calendar year, with not more than 540 days between successive inspections
Publicly accessible vehicular bridges – regular intervals not to exceed twenty-four months
For the remainder of the data elements there is no expiration date for source documents
Generating Source Documents Downloading or printing FIMS data for multiple data
elements and having several individuals sign off on it is not valid
If there is no historic source document for a particular data element, and it cannot be generated elsewhere, it is acceptable to develop a table from data currently in FIMS for that particular data element, have the Site’s Subject Matter Expert (SME) review the data, make needed corrections and sign and date the table indicating that he/she certifies the data is accurate The table can only contain data for one SME and the SME
must personally review and certify the data as correct. This can be accomplished by signing and dating the table or by sending an email with the table/spreadsheet attached. Example: Main Location, City, State, County, Zip Code and Congressional District may
be combined within one table
Generating Source Documents (continued)
This procedure cannot be used for:
Size (Gross Sqft, Rentable Sqft, Primary Qty, Acreage)
Annual Actual Maintenance
Asset % Utilized Sustainability
RPV Repair Needs
Deferred Maintenance Inspection Date
Annual Operating Cost Bridge Safety Inspection
Gathering Source Document
As you are updating FIMS, you should be receiving hard copy/electronic documentation for your source data There should be no need to regenerate Source
Documents just prior to the validation Source documents should be gathered and
organized prior to the kickoff of the FIMS Validation Consider organizing source documents into file
folders/binders by data element sorted by Property ID for easy retrieval during the validation
Gathering Source Document (continued)
All source documents must be finalized Signed and dated by the Subject Matter Expert
If not, is considered an unacceptable source document Emails printed with dates/To/From info and
attachments referenced by file name are acceptable For clarification purposes, printed email attachments
should include the file name and date
Source Documentation Worksheet
Used to identify the Subject Matter Expert for each validated data element
The Source Documentation Worksheet can be found at http://fimsinfo.doe.gov/data_validation.htm
Source Documentation Worksheet
To be completed for your site at least 1 week prior to your scheduled FIMS validation
Ensure Subject Matter Experts expect to be contacted should there be any questions
FIMS Data being Validated and What’s new for FY2015
FIMS Validation Data Elements and Performance Measures for DOE Owned Assets Ownership Usage Code Property Type Status Size
Gross Sqft (buildings and trailers) Primary Qty (OSF)
Repair Needs Asset % Utilized (buildings and trailers)
Replacement Plant Value Deferred Maintenance Operating Cost (site level) and
Annual Actual Maintenance (asset level)
Mission Dependency Historic Designation
Using Organization Main Location Location City/State/County/Zip
Code Location Congressional District Excess Indicator Estimated Disposition Year Total No of Federal Employees
(buildings and trailers)
Total No of Contractor Employees (buildings and trailers)
Total No of Other Personnel (buildings and trailers)
Total No of Occupants (buildings and trailers)
Outgrant Indicator Initial Acquisition Cost Total Adjustment Cost
GREEN – Performance Measures
FIMS Validation Data Elements and Performance Measures for DOE Owned Assets (continued)
Roads – Public Access Miles Roads – Public Access Lane
Miles Inspection Date Hours of Operation Per Week Sustainability (buildings and trailers)
Year Built Year Acquired Meters Determination Date for DOE
Disposal GSA Notification Submitted GSA Notification Accepted
GREEN – Performance Measures
FIMS Validation Data Elements and Performance Measures for DOE Leased and GSA Assets Ownership Usage Code Property Type Status Size
Rentable Sqft (buildings and trailers) Primary Qty (OSF)
Asset % Utilized (buildings and trailers)
Replacement Plant Value* Annual Actual Maintenance Mission Dependency Using Organization Main Location Location City/State/County/Zip
Code Location Congressional District
Estimated Disposition Year* Total No of Federal Employees
(buildings and trailers)
Total No of Contractor Employees (buildings and trailers)
Total No of Other Personnel (buildings and trailers)
Total No of Occupants (buildings and trailers)
Outgrant Indicator* Inspection Date* Hours of Operations per Week Sustainability* (buildings and trailers)
Meters* Expiration Date Annual Rent Lease Authority* Year Built*
GREEN – Performance Measures * - Validated for DOE Leased only
FIMS Validation Data Elements and Performance Measures for Land Assets Ownership Usage Code Property Type Status Size
Acreage
Mission Dependency Historic Designation (owned only)
Using Organization Main Location Location City/State/County/Zip
Code Location Congressional District
Excess Indicator Estimated Disposition Year Outgrant Indicator Expiration Date (leases only)
Annual Rent (leases only)
Lease Authority (leases only)
Year Acquired
GREEN – Performance Measures
FIMS Validation Data Elements for Archived/Disposed Assets
Disposition Date Disposition Method Net Proceeds
New Data Elements for FY15 Validations
Let’s take a look ….
Repair Needs
Definition: The estimated cost to restore a real property asset’s component system failures noted during a condition assessment survey to a state substantially equivalent to the most recently configured capacity, efficiency, or capability as required by the mission
Validated for buildings, trailers and OSF Must be greater than zero Must be equal to or greater than the Deferred
Maintenance
Repair Needs (continued)
Update frequency: Annually How to check: CAS reports as the asset level;
CAIS report Source Documentation is required for all values,
including $0 (zero) Anomalies: cannot equal zero or be less than
Deferred Maintenance
Asset % Utilized (Performance Measure)
Definition: That portion of an asset in use Utilization is independent of Status and must be reported for all
FIMS Status designations Facilities that are fully shutdown will generally have a Utilization of
zero. However, because the reported “Status” is the predominate status, a facility that is 51% shutdown and 49% operating would be properly reported as ‘Shutdown” but the Utilization would be 49% (if the operating portion is fully utilized).
Utilization is not a reflection of space assignment but rather a measure of how ‘full’ the space is. That is how utilized the space is. As an example, if a building with four floors is fully assigned/charged to an organization but one floor is empty with the other three floors fully utilized, the Utilization would be 75%.
For programmatic facilities such as laboratories or accelerators, the Utilization can be considered to be 100% if the mission requires 100% of the facility, even though there may be times when the facility is not fully utilized, or even unutilized.
Asset % Utilized (Performance Measure) (continued)
Utilization = (Utilized GSF of an asset / Total GSF of an asset) X 100 (expressed as a percentage in FIMS)
RPAM requires annual utilization surveys Only validated for buildings and trailers Update frequency: Annually How to check: examine process for measuring utilization, verify
that reported utilization is calculated from this process and visit a sample of assets to do a reasonableness check
Anomalies: Utilization should not be a reflection of space assignment
Initial Acquisition Cost
Definition: Total final project cost for buildings, trailers and OSF
Validated for DOE owned buildings, trailers and OSF Only being validated - will not be scored on the
Validation Scorecard Update frequency: As needed How to check: contract document, financial report
Total Adjustment Cost
Definition: The total of all capital adjustments / improvements to the property
Validated for DOE owned buildings, trailers and OSF Only being validated - will not be scored on the
Validation Scorecard Update frequency: Annually How to check: contract document, financial report
Inspection Date Definition: The date of the final Condition
Assessment Survey (CAS) during the fiscal year Verify date is within a 5 calendar year period (i.e.,
Inspection Date = 6/1/2010 must be reinspected by 6/1/2015) If a variance, must be denoted for Deferred Maintenance
and Repair Needs on the Data Validation Reporting from Validated for buildings, trailers and OSF
Inspection Date (continued)
Update frequency: Annually How to check: CAS reports as the asset level;
CAIS report
Deficiencies need to be updated annually,
inspections are to be performed every 5 years or less
Roads – Public Access Miles
Definition: Record the subset of the OSF Primary Quantity miles that are public access. Enter 0 (zero) if no publicly accessible roads.
Source documentation is required for all values, including 0 (zero)
Validated for OSF assets with Usage Code = 1729 – Primary Roads, 1739 – Secondary Roads, 1749 – Tertiary Roads
Update frequency: As needed How to check: Subject Matter Expert in charge of
site infrastructure support
Roads – Public Access Lane Miles
Definition: Record the quantity of lane miles that are public access. Enter 0 (zero) if no publicly accessible roads.
Source documentation is required for all values, including 0 (zero)
Validated for OSF assets with Usage Code = 1729 – Primary Roads, 1739 – Secondary Roads, 1749 – Tertiary Roads
Update frequency: As needed How to check: Subject Matter Expert in charge of
site infrastructure support
Hours of Operation Per Week
Definition: An approximation of the “lights on” hours for a building
Validated for buildings and trailers Update frequency: As needed How to check: building/facilities manager Anomalies: asset is in a shutdown status with
Hours of Operation set to 60 hours per week or more
GSA Notification Submitted
Definition: Date that represents when DOE submitted a Report of Excess (ROE) to GSA and is pending acceptance by GSA
Validated for DOE owned buildings, trailers, and OSF Update frequency: Annually How to check: signed transmittal memo or email
forwarding SF 118 to GSA
GSA Notification Accepted
Definition: Date that represents DOE has received an acceptance of the Report of Excess (ROE) from the GSA disposal office
Validated for DOE owned buildings, trailers, and OSF Update frequency: Annually How to check: GSA notification
Year Built Definition:
For buildings and trailers New construction, the fiscal year (YYYY) that the
building/trailer is accepted for beneficial occupancy Acquiring an existing asset, the fiscal year the building/trailer
was constructed For OSF the calendar year (YYYY) construction of the
structure was completed for the following Usage Codes 1468 Public Access Bridges (Trains) 1469 Controlled Access Bridges (Trains) 1768 Public Access Bridges (Vehicular) 1769 Controlled Access Bridges (Vehicular)
Validated for DOE owned and DOE leased buildings, trailers and OSF with Usage Codes 1468,1469,1768 and 1769
Year Built (continued)
Only being validated - will not be scored on the Validation Scorecard
Update frequency: As needed How to check: date of Beneficial Occupancy letter
or other document that established year built; Bridge Safety Inspection report
Year Acquired
Definition: For buildings, trailers and OSF, the fiscal year (YYYY)
acquired by DOE For land, the year represents the fiscal year (YYYY) of the
earliest land parcel acquisition Validated for DOE owned buildings, trailers and OSF
and DOE owned, DOE leased and withdrawn from public domain land
Only being validated - will not be scored on the Validation Scorecard
Update frequency: As needed How to check: contract document, financial report,
land deed or lease agreement, “Public Land Order” effective date for withdrawn from public domain land
Disposition Date
Definition: The date the disposal action was completed
Disposition Method Disposition Date value
8 – Federal Transfer Date of letter of transfer
10 – Demolished Date demolition is complete
17 – Other Disposition Archive date
All Public Benefit Conveyance (CF,HA,HE,HM, LW,NS,PA,PF,PR,SH,WC)
Date of assignment letter to sponsoring agency or deed date to grantee
SN – Negotiated SaleSP – Public Sale
Deed date
TM – Early Termination/ Cancellation
Termination/Cancellation date
XP – Expiration/Cancellation Expiration/Cancellation date
Disposition Date (continued)
Validated for Archived/Disposed assets Update frequency: Static How to check: Site Subject Matter Expert in charge
of asset disposition, i.e. Real Estate, Disposition Manager, Accounting
Disposition Method Disposition Date value
8 – Federal Transfer Date of letter of transfer
10 – Demolished Date demolition is complete
17 – Other Disposition Archive date
All Public Benefit Conveyance (CF,HA,HE,HM,LW,NS, PA,PF,PR,SH,WC)
Date of assignment letter to sponsoring agency or deed date to grantee
SN – Negotiated SaleSP – Public Sale
Deed date
TM – Early Termination/ Cancellation
Termination/Cancellation date
XP – Expiration/Cancellation Expiration/Cancellation date
Disposition Method
Definition: The method in which the real property asset left the
Department’s inventory
8 – Federal Transfer HM – PBC – Historic Monuments
SH – PBC – Self-help Housing
10 - Demolished LW – PBC – Law Enforcement and Emergency Mgmt Response
SN – Negotiated Sale
17 – Other Disposition NS – PBC – Negotiated Sales to Public Agencies
SP – Public Sale
CF – PBC – Correctional Facility Use
PA – PBC – Public Airports TM – Early Termination/ Cancellation
HA – PBC – Homeless Assistance
PF – PBC – Port Facilities WC – PBC – Wildlife Conservation
HE – PBC – Health or Educational Use
PR – PBC – Public Parks and Recreational Area
XP – Expiration/ Cancellation
Key: PBC – Public Benefit Conveyance
Disposition Method (continued)
Validated for Archived/Disposed assets Update frequency: Static How to check: Site Subject Matter Expert in charge
of asset disposition, i.e. Real Estate, Disposition Manager, Accounting
8 – Federal Transfer HM – PBC – Historic Monuments SH – PBC – Self-help Housing
10 - Demolished LW – PBC – Law Enforcement and Emergency Mgmt Response
SN – Negotiated Sale
17 – Other Disposition NS – PBC – Negotiated Sales to Public Agencies
SP – Public Sale
CF – PBC – Correctional Facility Use PA – PBC – Public Airports TM – Early Termination/ Cancellation
HA – PBC – Homeless Assistance PF – PBC – Port Facilities WC – PBC – Wildlife Conservation
HE – PBC – Health or Educational Use
PR – PBC – Public Parks and Recreational Area
XP – Expiration/ Cancellation
Net Proceeds
Definition: For assets with a Disposition Method set to
SP – Public Saleor
SN – Negotiated Sale Report the proceeds less disposal cost
TM – Early Termination/ Cancellation For Early Termination/Cancellation, report the cost avoidance
from early termination less the costs incurred to prepare the leased property for return to the owner
Net Proceeds can be zero or negative in cases where the disposal cost exceeds proceeds
Net Proceeds (continued)
Validated for Archived/Disposed assets Update frequency: Static How to check: Site Subject Matter Expert in charge
of asset disposition, i.e. Real Estate, Disposition Manager, Accounting
Bridge Safety Inspection Verification
Bridge Safety Inspection Report Latest Safety Inspection will be verified for OSF
usage codes: 1468 Public Assess Bridges, Trains 1469 Controlled Access Bridges, Train 1768 Public Access Bridges, Vehicular
Required Bridge Safety Inspection Date intervals: Controlled access and publicly accessible train bridges – one
inspection per calendar year, with not more than 540 days between successive inspections
Publicly accessible vehicular bridges – regular intervals not to exceed twenty-four months
Bridge Safety Inspection Verification (continued)
Required when Status is: 1 – Operating 2 – Operational Standby 6 – Operating Pending D&D 7 – Operating Under an Outgrant
New for FY15 – Required to validate Location Zip Code Year Built
Update frequency: Required to be updated within 30 days of inspection completion
How to check: Bridge Safety Inspection report
Data Elements that were removed from the Validation for FY15:Secondary Quantity (OSF only) Cool Roof – Not Economically
Feasible
Cool Roof – Total Roof Projected Area (GSF)
Sustainability – Compliance Approach
Cool Roof – Vegetative Area (GSF)
Sustainability – USGBC Project ID
Cool Roof – Reflective Area (GSF)
Sustainability – Certification Level Received
Cool Roof – Photovoltaic Area (GSF)
Acquisition Method (Land only)
Cool Roof – Planned Complete Cool Roof Date
Process Verification
Processes
It is essential to look at the processes at a site The methodology for capturing the data A site’s procedures for collecting the data
Source documents are important but so are the source processes The Validation is more than just matching the FIMS
value to the Source Document value Need to be documented
Process to look for …
Deferred Maintenance and Repair Needs – Is it inspection based? What exactly is the process?
Actual Maintenance – Is it captured at the asset level?
Examine/explain the process Inspection Date –
Verify the inspection cycle is 5 years or less
Process to look for … (continued)
Asset % Utilized - Is it based on an asset level annual survey?
Explain the process Replacement Plant Value –
Are FIMS models used? If contactor developed, does the calculation take
into account current standards of construction? What is the process?
Verify that CPV is only used for OSF’s and new assets prior to the first CAS inspection
Process to look for … (continued)
Site Factor If the default 1.586 is not being used
What process was used to develop the Site Factor? Are the calculation sheets available?
Operating Cost - Is the Site level cost based on actual
invoices/contracts? If site allocated the asset level cost, is the process
documented? Explain the process
Ensure it is based on sound business practices
Desktop Validation using the Data Validation Reporting Forms
Data Validation Reporting Forms
Forms provide the basis of the validation effort
Completed for each of the assets to be validated DOE owned, plus the site level Operating Cost
and the Bridge Safety Inspection verification DOE leased and GSA Land Archived/Disposed
Data Validation Reporting Forms (continued)
Provides structure for Identifying difference between the FIMS value and
the Source Document value Commenting on issues and observations
DOE Owned Data Validation Reporting Form
DOE Owned Data Validation Reporting Form (continued)
DOE Leased and GSA Data Validation Reporting Form
DOE Leased and GSA Data Validation Reporting Form (continued)
DOE Land Data Validation Reporting Form
DOE Disposition Data Validation Reporting Form
Data Validation Reporting Forms
Care should be taken to not modify the Data Validation Reporting Forms. There are a multitude of macros and formulas that allow the forms to populate and calculate the data.
Also don’t try to move or insert pages within the forms, this will break the processing setup in the forms
“Bridge Inspection” page of the DOE Owned Data Validation Form
Provides the following identifying information on each bridge asset:
Property ID Status
Property Name Primary Qty
Usage Code Desc Location Zip Code
Safety Inspection Date Year Built
“Bridge Inspection” page of the DOE Owned Data Validation Form (continued)
The following information is to be populated from the Bridge Safety Inspection source document Date of Inspection from Safety Inspection Report Date within 1 yr - Trains or within 2 years - Public Access
Vehicular* (Yes/No) Date within:
One calendar year, with not more than 540 days between successive inspections for controlled access and publicly accessible train bridges
Regular intervals not to exceed twenty-four months for publicly accessible vehicular bridges
Safety Inspection Report signed and dated by certified bridge inspector/team lead (Yes/No)
Safety Inspection supports operational (FIMS) status of bridge (Yes/No) FIMS Status = ‘Operating’, the source document should reflect ‘Operating’
“Bridge Inspection” page of the DOE Owned Data Validation Form (continued)
The following information is to be verified from the Bridge Safety Inspection source document Usage Code Description Location Zip Code Year Built
Correction to Discrepancies
Discrepancies discovered during the validation may be corrected during the scheduled validation period to improve the score To improve the score for a particular data element, all
records for the site for that particular data element must be reviewed and corrected All assets must be reviewed against a correct source document FIMS must be updated with correct values Change the extracted data on the Data Validation Reporting Form Must be completed within the Validation schedule
Only reviewing and correcting the assets being validated is not sufficient to improve the score for the data element
The Walkthrough Process
The Facilities Walkthrough
Although the validation is largely a ‘paper validation’; it is critical that the review team see and touch some of the assets to verify usage, utilization and overall condition of the facility
The walkthrough is not a condition assessment survey, but more of a reality check
Some facility knowledge is necessary but the walkthrough relies more on observation and communication skills
The walkthrough is performed in the later part of the validation process, after the desktop validation
The Facilities Walkthrough (continued)
After the Data Validation Reporting Forms are generated, pick a few of the assets being validated to visit Recommend that a walkthrough be conducted on 30%
of the assets selected for the validation For a site validating 25 DOE owned assets and 5 DOE
leased and GSA assets, you should walkthrough 7-9 buildings/trailers/OSF
Include DOE leased assets, GSA assets and land assets
No hard rules on which to select to walkthrough This is not a detailed facility inspection, but an
overview to verify actual conditions of a real property asset are consistent with source data and FIMS data
Expect a walkthrough to take no more than 10 minutes per facility
The Facilities Walkthrough (continued)
Drive by as many of the remaining assets as practical in the sample sets Verify:
The asset exists Property Type Status
Drive by every train and vehicle bridge to verify Usage Code and Status
Drive by 100% of the previous location of the Archived/Disposed assets that are being validated
The Facilities Walkthrough (continued)
To help confirm that all existing real property assets are recorded in FIMS, conduct an existing asset check During the walkthroughs record the Property ID of 10
assets (5 buildings/trailers and 5 OSF) that are not part of the sample sets to confirm that these assets are in the FIMS database. At the conclusion of the walkthrough verify the 10 assets exist in FIMS.
The Facilities Walkthrough (continued)
Prior to walkthrough: (continued):
Generate reports from FIMS on all facilities to be visited/viewed For buildings,
DOE owned – #001 Owned Building Complete Information Report
DOE leased – #019 Leased Building Complete Information Report
GSA Owned and Leased - #072 GSA Occupancy Agreements Complete Information Report
The Facilities Walkthrough (continued)
Prior to walkthrough: (continued):
Generate reports from FIMS on all facilities to be visited/viewed For trailers,
DOE owned – #050 Owned Trailer Complete Information Report
DOE leased – #052 Leased Trailer Complete Information Report
For OSF, DOE owned – #040 Owned OSF Complete Information
Report DOE leased – #044 Leased OSF Complete Information
Report
The Facilities Walkthrough (continued)
Prior to walkthrough: (continued):
Generate reports from FIMS on facilities to be visited/viewed (continued)
For Land, DOE owned – #030 Owned/Withdrawn Land Complete
Information Report DOE leased – #035 Ingrant Land Complete Information
Report For Bridges,
#068 Bridge Safety Inspection Report to PDF format For Archived/Disposed
Print the sheet labelled “XRef” from the DOE Disposition Validation Reporting Form
The Facilities Walkthrough (continued)
Sample FIMS #001 Owned Building Complete Information report
Red arrows identify items that can be verified on walkthroughs
Scorecarding
Scorecarding
Scorecard reporting is: Compact and conveys results quickly To be finalized during the validation Focuses attention on the facts and findings
FY2015 FIMS Validation
Four scorecards will be generated One for DOE owned buildings, trailers and Other
Structures and Facilities (OSF) One for DOE leased buildings, trailers and Other
Structures and Facilities (OSF) and GSA Owned and Leased Buildings
One for Land – DOE owned, DOE leased and withdrawn from public domain
One for Archived/Disposed assets
Sample Scorecard
Score Green Yellow Red
Data ElementNo more than a 5% frequency of variance
Greater than 5%, but no more than 10% frequency of variance
Greater than 10% frequency of variance
Explaining the Scorecard At the data element level, red, yellow and green ratings are
assigned based on the frequency of variance percentage:
Data elements which are validated to source documents but are outside accepted definitions or conventions are assessed red with an explanation Example: RPV matches between FIMS and the source document but is
considered to be a variance because through discussion with the site, RPV is determined to really be CPV for all the site’s owned buildings. As a result, this data element would be red.
Rating of Status
To get the green in status: All performance measures are green
RPV, Deferred Maintenance, Asset % Utilized, Operating Cost (site level) and Annual Actual Maintenance
No more than two of the remaining elements are yellow
Rating of Status
A rating of yellow is received if: If one or more performance measure is yellow
RPV, Deferred Maintenance, Asset % Utilized (buildings/trailers
only), Operating Cost (site level) and Annual Actual Maintenance Or, no more than one of the remaining elements is red
An overall status rating of red is received If one or more of the performance measures result in a red rating
RPV, Deferred Maintenance, Asset % Utilized (buildings/trailers
only), Operating Cost (site level) and Annual Actual Maintenance Or, more than one of the remaining elements are red
Rating of Status Chart
FIMSScorecard
Status Score
PerformanceMeasures *
Remainder of Validated Elements
Notes
GreenAll performance measures green and…
No more than two of the remaining elements yellow
Preferred level of data quality
Yellow
One or more performance
measure yellow or…
No more than one of the remaining elements red
Minimal acceptable level of data quality
Red
One or more performance
measure red or…
More than one of the remaining elements red
Unacceptable data quality
*Performance Measures: Replacement Plant Value, Deferred Maintenance, Asset % Utilized, Annual Operating Costs and Annual Actual Maintenance
Sample Scorecard
Status Rating of RED If a Site’s Status rating is Red, the site is required to
develop a Corrective Action Plan (CAP) within 15 days of the validation A copy of the CAP is to be delivered to the Site Office and HQ Program
Office The HQ Program Office will forward a copy of the CAP to OAPM and
oversee the execution of the plan
The Site will also have to provide a schedule/date to the Site Office/HQ Program Office for the revalidation as a result of the red score Revalidation required within 60 days or before August 1st of the current
fiscal year Revalidation is only required for those data elements that scored either
Red or Yellow
Corrective Action Plan (CAP) Sample available on FIMS website at
http://fimsinfo.doe.gov/data_validation.htm Usually just the scorecard with a few additional
columns to identify: Correction plan of action Point of contact and contact information Estimated date of completion Actual date of completion
Rating of Progress A progress rating is important because the status
rating may not tell a whole story Progress rating is more subjective and should consider:
Statistical improvement since last validation Establishment / meeting the goals of a corrective action plan
The progress rating is a leading indicator and a way to indicate positive (negative) momentum that will yield status improvement (decline)
Rating of Progress (continued)
Red – indication that red status will not improve or yellow status will decline Lack of attention and urgency Corrective action plan not being met
Yellow – indication that limited, positive efforts are being taken that may improve the data Insufficient attention being given Corrective action plan in place but progress is slipping
Green – results-oriented plan in place with goals and necessary resources Clear level of attention and demonstrated sense of urgency Frequency of variance is going down Typically, a site that is green is status by default is green in
progress
Rating of Progress Chart
FIMS Scorecard
Progress Score
Improvement of Data
Goals of the Corrective Action Plan
Notes
Green
Status is green or significant improvement since last validation
Goals established and being met or no plan needed
If Status is green or yellow, no correction plan is required and progress is typically green
Yellow
Some improvement in data quality
since last validation
Progress is slipping
Red
Minimal improvement in data quality since last validation
Goals for improvement not being met
Sample Scorecard
Score Green Red N/A
Bridge Safety Inspection Verification
All required data is verified as correct
Discrepancies noted with required data or asset visit
Site does not have bridges
Rating of Bridge Safety Inspection Verification
Rating of Existing Asset Check
After verifying that the 10 recorded properties from the walkthrough are in the FIMS database, change the Existing Asset Check rating box on the scorecard as follows: Green if all 10 assets are found in FIMS Red if one or more of the 10 assets are not
in FIMS
Disposition Scorecard Rating
After verifying the FIMS values with the Source Document values and confirming that all Archived/Disposed assets no longer exist, update the Status as follows: Green if all disposition data elements are
green and all assets have been confirmed as being disposed
Red if any disposition data element is red or an asset is found to still be on the Site
Finalizing the Scorecard On the Scorecard sheet
Update the header information for Site Name Program Dates of Validation
To record the ratings, simply type Green, Yellow or Red in the appropriate cell This will color the cell background
appropriately Add Notes/Recommendations and General
Comments
Planning the Validation
Frequency and Timing
Validations are to be conducted annually for all sites If a Site’s scorecard Status is red however, the Site is
expected to conduct revalidations in 60 days or before August 1st
Revalidation will only be required for those data elements that scored either Red or Yellow
Frequency and Timing (continued)
The FIMS Validation guidance recommends that validations should be completed between January 15th and June 30th of each fiscal year Provide HQ Program Office with planned Validation date
by 11/30 of each fiscal year Validation should not occur between August 1 and
December 15 which is the annual update cycle for FIMS data
Validation Schedule and Quality Assurance Reviews Site’s planned validation dates may be found on the
Validation Schedule Found on the FIMS website at
http://fimsinfo.doe.gov/data_validation.htm
Quality Assurance Reviews Performed by OAPM/Headquarters Programs Occur every 5 years unless otherwise needed/requested To verify consistency and to ensure validations are conducted in
accordance with the guidance Sites to be visited are highlighted on the Validation Schedule
Retention of Information
What should be kept? Completed Data Validation Reporting
Forms which includes the Scorecard Source Documents
How long? No less than five years
Allows trending and tracking Who should keep?
Site HQ Program Office and OAPM Site should email completed Data
Validation Forms to the appropriate HQ program office and OAPM
Validation Guidance and Website
FIMS Validation Materials and Guidance Validation materials are available from the FIMS website at
http://fimsinfo.doe.gov/data_validation.htm
The FIMS FY15 Data Validation Guidance Memo issued by the Office of Acquisition and Project Management (OAPM), DOE can also be found on the FIMS website
FIMS Data Validation Update for FY15
Thank you
Presented by: Mark Gordy, GCI and Gayle Smith, Stargates Inc
Recommended