Team

Narrative

Mind

Team Info

The Narrative Mind team contains experts in software engineering, social media design, and

web-based information operations (IO). We seek to develop tools that will optimize discovery

and investigation of communication trends on social media.

Weekly Total: 10 interviews

Users: 16

Experts: 45

Buyers: 5

Cumulative Total: 66 interviews

OTA Process -- Next Steps

OTA Summary

● Prototype projects only.

● Small businesses or non-traditional

defense contractors participate to a

significant extent.

● More negotiable IP terms

● Initial Award ~$5M

● Multiple phases

Phases Post Initial Award● Iterative process (detailed view, next slide)

● Number of phases is flexible.

● Amount of money is flexible.

● Color of money is flexible.

● Scale is bounded by “production” levels.

Can’t be used to “reduce competition” or

“increase production.

Alternatives Briefly Compared

● SBRIs are smaller in scale and require

more administrative overhead.

● DARPA Cyber FastTrack is deprecated

but had a similar idea to OTA.

● CRADA is designed for research sharing,

not really prototype efforts.

OTA Iterative Evaluation Process

No overarching regulations or timelines.

“Get back to me in a year and I’ll let you how it goes.”

Example: Parsons, Critical Stack & Wolf Den

Innovation Challenge Initial Award Testing Evaluation

~$5M

1. Industry Day

2. Reqs. Synopsis

3. Request White Papers

4. Received 13 papers

5. Four proposals selected.

6. “Demo” replaced by

Technical Discussion.Original

requirements

synopsis

modified.

1. ACT office has

personnel working with

testers from all ranks.

2. Army Cyber Battle Lab

involved for

integration/concepts.

Budget Adoption

??????????

Final acquisition may

be expedited if

requirement money is

already set aside in

budgets, e.g. RDT&E

1. Problem

requirements are

changed iteratively.

2. Phase objectives

and timeline are

flexible.

Variants of OTA Processes

OTA processes can vary significantly.

● Type of Material/Resource

○ Different PEO

○ Different Consortium

● Different Funding Model

○ Prototypes?

○ Investment?

● Evaluation Style

○ Dependent on funding?

Example: Army Battery Technology circa 2003 (?)

OTA process similar to In-Q-Tel model.

● Army Venture Capital Initiative created non-

profit venture fund OnPoint

● No prototype

● Investment based

● RDT&E money and S&T money○ Relatively larger budgets and lasts longer.

● ~$25M initial fund?

● Future customer privileges?https://www.ciaonet.org/attachments/11954/uploads

http://www.businesswire.com/news/home/20041111005159/en/PowerGenix-Announces-Closing-Series-Financing-OnPoint-

Technologies

Hypotheses ❏ Our narrative units may exist in other commercial tools

❏ If not, make the introduction between ARCYBER and relevant tools

❏ Communication links between problem awareness and problem response are weak

Experiments ❏ Talk to commercial tools about their tools, direct comparison to our narrative units

Results There is overlap between our narrative units and other commercial tools on social network

analysis

Actions 1. Set up “demo” day to show ARCYBER a selection of commercial tools with capabilities that

seem to fit their social media network analysis needs

2. Use the commercial tools demo’d as a foil to understand what ARCYBER needs and doesn’t

need

Customer Discovery

Commercial Tools’ Value

- “Intelligent” content filtration and

categorization systems

- Powerful data visualization

- Geospatial integrations

- Cross-platform information integrity

MVP - “Demo Day”

Research/Academic Tools’ Value

- Novel scraping and aggregation

techniques

- Domain expertise in counterterrorism

- Human intuition for rooting out

narratives

Wednesday, May 4 with ARCYBER sponsors in person

MVP - 3 Types of “Narratives” Outputted by Our System

A list of hashtags that

appear with the same

message with extreme

frequency.

Positive or negative

opinion of geographic or

demographic population.

Narrative Type #1 Narrative Type #2Narrative Type #3

Displayed

to Analysts

Analysts associate

each “narrative unit”

with a real-world,

human-readable

description of a

narrative.

Gain creator: Software automatically

tracks activity of a real-world narratives by

tracking the activity of associated “units”.

Real World NarrativeLong-form, unstructured human description or

collection of info.

A ranked list of’ most

popular multi-media

content over time

Pain reliever: Social media firehoses

scanned for list of narrative units that match

predefined “schemas”, ranked by frequency.

Problems with this MVP:● What are the right “units”?

● How is a narrative’s overall growth

calculated from the multiple units with

different sizes? (e.g. relative

“weights?)

Benefits of this MVP:● Aside from schema, specific units

don’t need to be defined a priori.

● Model allows retroactive reindexing of

a single narrative’s growth based on

new associations.

● Builds a “training set” for more

sophisticated efforts.

Responses to Narrative Units Presented Previously

A ranked list most

popular content

over time

A list of

hashtags that

appear with the

same message

with extreme

frequency.

Positive or negative opinion among

geographic or demographic

population.

Narrative Type #1

Narrative Type #2

Narrative Type #3

E.g

Popula

rity

with u

sers

Sentiment analysis is thought

to be too crude to provide

useful input. May pollute

signal-noise.

Unit #2 “popular content”

(commonly described as

media like photos or videos)

seemed to be the most

credible. Unit #1 co-occurring

hashtags might provide more

granularity and context than

just single hashtag trends.

All the Hypotheses

Global

Tweet-level

Awareness Response

Adversary IO Org Chart

No baseline for monitoring/aggregating

use of tech; cyber targeting

Automatic Narrative Detection

Language/culture experts aren’t able

to work at scale

Virality Predictor

Understand which narratives are the most salient

Important Event Predictor

Preempt events

Counter-Narrative Generator

DoD/Gvt. social media

presence is weak

Persistent ID-Alias tracker

Account bans and multiple aliases across

different networks make IDs hard to track

Filesharing Site Scraper

Need more cached information

access

Expedited Content Categorization

Scale of social media makes manual efforts

painful

Bot Detector

Can’t determine actual scale of

organization’s following and support.

Mission Model Canvas

- Categorize social media posts

and users by content for

monitoring and tactical purposes.- Gnip/Twitter/Facebook

- CrowdFlower,

Samasource, or

Mechanical Turk

- Pre-existing social

media service and micro-

labor aggregators

- Third-party access

platforms for social

media,

-Data visualization

-Content analysis

platforms (Sens.ai,

Leidos)

Primary

ARCYBER

-Bg. General (decision

maker)

-MAJ/LTC/COL

(operational plan)

-Analysts/Operators

(actionable insights)

COCOMs

-General (decision

maker)

-MAJ/LTC/COL

-Analyst/Operator

Secondary

Political Campaigns

-Campaign managers

-Supporters

Consumer Brands

-CMO

-Public Relations Team

- Optimize workflow for

social media analysts.

-Deliver insights to

commanders about

online environment.

-Insights into tactical

responses against

specific targets

broadcasting narratives

-Detect in real time

-Early warning - Architecture that can

support massive

concurrent data

aggregation and

analysis.

- Customized UI

- Testing with analysts

- MechanicalTurk or crowdsourcing labor (microtasks)

- UI Development/Testing with ARCYBER analysts.

- Software Development

- Access to Twitter

firehose or API

- Local language

speaking crowdsourcing

staff.

- Accurate testing for

intercoder reliability

- ARCYBER: Bg. General,

LTC, Strategic Initiatives

Group, OTA, Purchasing PMs,

End-User operator)

-COCOMs: OTA, Operators,

Purchasing PMs

-Political Campaign:

Opposition research team, ???

-Private Sector: CMO, ???

Beneficiaries

Mission AchievementMission Budget/Costs

Buy-In/Support

Deployment

Value

PropositionKey Activities

Key Resources

Key Partners

Value Proposition Canvas - ARCYBER

Products

& Services

Web/Desktop

Application Provide high-level

overview of groups,

update research

Customer

Jobs

- New mission priorities.

- Little familiarity with overall

environment.

- No repository for high-level

learnings

Gains

Pains

Gain

Creators

Pain

Relievers

- New opportunities

-Illustrates evolution of info

dissemination networks

- Provides insight for

mapping “online terrain”

Visualize information about

how organizations are

using social media

ARCYBER - Operator/Analyst (Actionable Insights)

Value Proposition Canvas - ARCYBER

Products

& Services

Web/Desktop

Application

Aggregate insights

from analysts,

compile into plan

passed to decision

maker

- New mission priorities.

- Proliferating/viral social

media platforms

- Urgency of issues ability to

prioritize

Customer

Jobs

Gains

Pains

Gain

Creators

Pain

Relievers

-Faster turnaround time with

better data gained from

information awareness software

and technology

ARCYBER - MAJ/LTC/COL (Operational Plans)

-Better data in informing

operational plans provided

to decision makers

-Reducing uncertainty in

operational planning

Value Proposition Canvas - ARCYBER

Products

& Services

Web/Desktop

Application

Act on reports and

plans generated

about various cyber

threats across globe- Little understanding of

ground-level nuances

- Little understanding of

proliferating platforms

Customer

Jobs

Gains

Pains

Gain

Creators

Pain

Relievers

- Organizational chart of

technology broken down by

functional purpose

- Timeline-based viewer

-Reduce uncertainty of

decision making

-Add methodological rigor

to ARCYBER’s

offensive/defensive

operations

ARCYBER - Brigadier General (Decision Maker)

Value Proposition Canvas - COCOM

Products

& Services

Web/Desktop

Application

Target audience

analysis, finding

opportunities to

optimize influence

operations

Customer

Jobs

-Loss of visibility on key

communicators (when

accounts shut down/opened)

- No repository for high-level

learnings

Gains

Pains

Gain

Creators

Pain

Relievers

-Auto-populated list & real-time

visualization of key accounts,

dissemination networks

-Allows more accurate

-Illustrates evolution of info

dissemination networks

- Provides insight for mapping

“online terrain”

Closure of visibility gap on

key accounts

COCOM - Operator/Analyst (Actionable Insights)

Value Proposition Canvas - COCOM

Products

& Services

Web/Desktop

Application

-Long term

operational planning

- Measuring

effectiveness

Customer

Jobs

- Unclear picture of how

some networks evolve,

adapt over time

Gains

Pains

Gain

Creators

Pain

Relievers

- Organizational chart of

technology broken down by

functional purpose

- Visualization of key accounts,

dissemination networks over

time

-Illustrates evolution of info

dissemination networks

- Provides insight for

measuring operations,

mapping “online terrain”

Greater awareness of

network evolution improves

planning, indicators of

operational effectiveness

COCOM - Maj/LTC/Col. (O4-O6) (Operational Plans)

Value Proposition Canvas - COCOM

Products

& Services

Web/Desktop

Application

- Judge potential

effect and risk of

operations

- Approve/reject

plans

Customer

Jobs

- Less familiarity with

internet as an operational

environment

- Needs high level summary

of how users use technology

Gains

Pains

Gain

Creators

Pain

Relievers

- Organizational chart of

technology broken down by

functional purpose

- Timeline-based viewer

-Illustrates evolution of info

dissemination networks

- Provides insight for

mapping “online terrain”

Visualize high-level

information about how

regional users use

technology.

COCOM - Brigadier General (O7) (Decision Maker)

Value Proposition Canvas - Political Campaign

Products

& Services

Web/Desktop

Application

-Identify opposition

narratives on SM

channels

-Craft responses to

respond w/short

turnaround

Customer

Jobs

-Limited lead time in

anticipating changes from

opposition campaigns

-New stories/themes can

derail campaign momentum

without a targeted response

Gains

Pains

Gain

Creators

Pain

Relievers

-Faster response time to

opposition’s campaigns

-Maintain the initiative in media

environment by rapidly

responding to threats

-Early detection of

emerging opposition

narratives on SM

-Insight into evolving

environment

Counter opposition

themes quickly, identify

newly spreading

rumors that could

undermine campaign.

Political Campaigns - Opposition Analysis/Campaign Director

Value Proposition Canvas - Consumer Product Company

Products

& Services

Web/Desktop

Application

-Monitor brand

integrity and

customer themes

-Identify and address

emerging problems

Customer

Jobs

-Identify themes about

corporate brand, pinpoint

problem areas early

-Avoid negative narratives

from becoming costly PR or

marketing problems

Gains

Pains

Gain

Creators

Pain

Relievers

-List of customer narratives

about brand that are being

posted online

-Proactive response to

emerging brand issues, better

customer satisfaction

-Early warning about

negative themes

-Identify problem areas,

protect corporate brand

from costly problems

-Protect brand integrity,

create greater

shareholder value by

addressing customer

concerns

Consumer Product Company - CMO/Marketing Organization