IAEAInternational Atomic Energy Agency

Enabling Collaborative Analysis: State Evaluation Groups, the Electronic State File,

and Collaborative Analysis Tools

Presenter: Chris Gazze

Authors: C. Gazze, C. Eldridge, T. Sköld, F. Rorif, Y. Feldman,

J. Murray, D. Calle, D. Gagne, B. Wilson

IAEA

Overview

Enabling collaborative analysis via:• State Evaluation Groups (people)

• Team Composition• Responsibilities

• Electronic State File (data)• Architecture and Technology (tools)

2

IAEA

Background

• Establishment of SEG format to: • Move to more collaborative working environment• Lay the ground for conducting integrated analysis• Make better use of knowledge and expertise across the

department

• Multi-country SEGs• To increase organizational efficiency (for example,

grouping several SQP States together) • To address common issues (for example, illicit trade)

3

IAEA

SEG Composition

• Multi-disciplinary & cross-departmental in composition• Inspectors• Subject matter experts: open source, nuclear material accounting,

trade, imagery, nuclear material balance evaluation, etc.• Fuel cycle expertise• Technologists• Safeguards concepts experts

• Team size varies depending on size and complexity of State nuclear programme• At least Country Officer (inspector), Alternate Country Officer, and

an SGIM open source analyst• Typical SEG size ranges from 3 to 10 members

4

IAEA

SEG Responsibilities: Approaches

• SEGs are responsible for establishing objectives and goals• SEG meetings draw on members’ expertise and the

information streams available to the Department to generate State-specific approaches

• Main Activities:• Performing Acquisition Path Analysis (APA)• Establishing Technical Objectives• Developing a State Level Approach (SLA)

5

IAEA

SEG Responsibilities: Plan & Monitor

• SEGs are responsible for planning and monitoring safeguards implementation

• Planning field and headquarters activities by drafting the Annual Implementation Plan (AIP)

• Monitoring AIP Implementation:• As individuals, SEG members participate in some of the

planned verification tasks according to their expertise• As a group, the SEG monitors implementation of the AIP

and adjusts plans as appropriate based on the outcomes of planned activities

6

IAEA

SEG Responsibilities: Analysis

• SEGs are responsible for ongoing analysis: • SEGs perform integrated analysis of information provided by States

and generated by verification activities • The analysis is performed to assess the correctness and

completeness of information declared by the State

• Example Activities:• Analysing State declarations• Analysing outcomes of in-field verification activities such as design

information verification, inspection, and complementary access• Examining environmental sampling results• Analysing outcomes of material balance evaluations• Analysing open source information, such as satellite imagery, trade

data, technical publications, news reports

7

IAEA

SEG Responsibilities: Reporting

• SEGs are responsible for Evaluating and Reporting:• SEGs analyse data, adjust plans for verification activities as necessary,

and report findings• On an annual basis, the SEG proposes a safeguards conclusion for the

State to IAEA senior management

• Reporting Mechanisms:• State Evaluation Report (SER): a living document that comprises the

Department’s primary compilation of safeguards relevant information and analysis. It is updated whenever new, safeguards relevant information becomes available

• State Evaluation Document (SED): a brief summary of the SEG’s analysis for the State, incorporating new information, identifying issues, and documenting progress on previous review committee recommendations

8

IAEA

Electronic State File: Common Platform

• Data collection and storage• Field Data: Inspection, DIV, CA, DA, ES• State Declared Data: NMA, AP, DIQ, correspondence, etc.• Open Source Data: publications, websites, images, trade, etc.

• Data analysis• Provides access to all safeguards relevant source data• Documents analytical work & allows linking to source data

• Review• Supports management, Divisional, and Departmental review

processes for state evaluation• Supports the process of drawing safeguards conclusions

9

IAEA

SIR

SEDSER

REPORTING

ANALYSIS

ANALYSIS &EVALUATION

COLLECTION

CA | INSP | DIV

FIELD OTHER SOURCES

ES | DA| OS| SIDIQ | AP | NMA

STATE SUPPLIED10

IAEA

State File

SGIMOperations Evaluation

FIELD OTHER SOURCESSTATE SUPPLIED

CACA

InspectionInspection

DIVDIV DIQDIQ

NMANMA

APAP OSOS

ESES

GEOGEO

DADA

SIR

SEDSER

ANALYSIS

11

IAEA

State File

SIR

SEDSER

ANALYSIS

Operations SGIMEvaluation

SLASLA

SEDSED

AIPAIPSER WIKISER WIKI

ANALYSIS

FacilityPagesFacilityPages

Intermediate Analysis

Intermediate Analysis

Ad Hoc AnalysisAd Hoc Analysis

CACA OSOS

ESES

NMANMA

GEOGEOInspectionInspection

DIVDIV

Corresp.Corresp.

DIQDIQ

APAP

DADA

SitePages

SitePages

Phy. Mod.Pages

Phy. Mod.Pages

Analytical ProductsAnalytical Products

SIR

12

IAEA

State File

Operations Evaluation SGIM

GEOGEOInspectionInspection

DIVDIV

Collaboration Spaces

SLASLA

SEDSED

AIPAIPSER WIKISER WIKI

ANALYSIS

Intermediate Analysis

Intermediate Analysis

Ad Hoc AnalysisAd Hoc AnalysisCorresp.Corresp.

APAP

DADA

SitePages

SitePages

Phy. Mod.Pages

Phy. Mod.Pages

Analytical ProductsAnalytical Products

NMANMA

FacilityPagesFacilityPages

OSOS

ESES

CACA

DIQDIQ

SIR

13

IAEA

Technologies

Collaborative Analysis is facilitated by:• A content management system that:

• Brings together and organizes source data for evaluation & analysis• Allows collaborative authoring of analytic products, e.g., wiki pages,

documents, and charts.• i.e: the Electronic State File

• A communication system, i.e., ISEmail• Database reporting and management systems (numerous)• Analysis platforms that provide specialized analytic

capability, for example: Palantir and the Geospatial Exploitation System (GES)

14

IAEA

Architecture

Ideally, all the analysis technologies integrate with each other, or, at a minimum, with the Electronic State File.

More integration

More cohesive user experience

More comprehensive analysis

15

IAEA

Safeguards’ Palantir instance has reached 14 Million objects of many types from ~10 sources. It’s still growing.

Palantir: All Sources – Large Scale

16

IAEA

Large-scale analysis means being able to work quickly with however many entities relate to your question.

In this case, thousands,

and 5,

at the same time.

17

IAEA

Palantir

And seeing the pattern in their relationships and activity over time…

18

IAEA

…and space.

Palantir

19

IAEA

Geospatial Exploitation System

Satellite imagery, map data, and imagery analysis reports accessible to all,

and protected by high-resolution access control.

20

IAEA 21

IAEA

Conclusions/Status

• SEGs• SEGs were created for every State several years ago

and are functioning well overall

• State File• Version 2 structure recently rolled out for all States and

upgraded to MS SharePoint 2013• But much data still needs migration into ISE

• Architecture and Technology• Some new tools in place (Palantir, GES) and training

has begun, but expanded use awaits data migration