24
/sin’fɒnjə/ Security Intelligence

Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

Embed Size (px)

Citation preview

Page 1: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

/sin’fɒnjə/

Security Intelligence

Page 2: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

Army Knowledge Online (www.us.army.mil) FM 2-0 INTELLIGENCE

/sin’fɒnjə/

Page 3: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

/sin’fɒnjə/The Intelligence Cycle

Direction

http://www.cni.es/es/queescni/ciclo/

Collection

Analysis

Dissemination

Page 4: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

/sin’fɒnjə/ This is NOT OSINTThis is Copy & Paste

http://tinyurl.com/pavtula

http://tinyurl.com/npegzok

http://tinyurl.com/q2ag2b9

February 26, 2014

Page 5: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

What is Intelligence?

Quite simply, intelligence is the information our nation’s leaders need to keep our country safe.

Our leaders, like the President, make policy decisions based on this intelligence.

/sin’fɒnjə/Intelligence (Kids’ Zone)

https://www.cia.gov/kids-page/6-12th-grade/who-we-are-what-we-do/what-is-intelligence.html

Page 6: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

• The generation of knowledge in support of decision makers

TroubleshootingAnticipation

• Intelligence is people (but not all people are intelligent):– Methodologies– Tools– Techniques

/sin’fɒnjə/Intelligence

Page 7: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

sheer volumen of information

volatile

time saving

gather structure

enrichclassify

store

real time

analyze

/sin’fɒnjə/Tools are Essential

integrate

Page 8: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

/sin’fɒnjə/

Storm Builder for Security Intelligence

Page 9: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

/sin’fɒnjə/Storm

“Apache Storm is a free and open source distributed realtime computation system. Storm makes it easy to reliably process unbounded streams of data, doing for realtime processing what Hadoop did for batch processing. Storm is simple, can be used with any programming language, and is a lot of fun to use! “

http://storm.incubator.apache.org/

Page 10: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

/sin’fɒnjə/Visual Programming

http://blog.interfacevision.com/design/design-visual-progarmming-languages-snapshots/

Page 11: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

/sin’fɒnjə/Module: Types

SPOUT BOLT DRAIN

Page 12: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

/sin’fɒnjə/Module: Types

SPOUT

“A spout is a source of streams in a computation. Typically a spout reads from a queueing broker such as Kestrel, RabbitMQ, or Kafka, but a spout can also generate its own stream or read from somewhere like the Twitter streaming API. Spout implementations already exist for most queueing systems.”

Page 13: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

/sin’fɒnjə/Module: Types

BOLT

“A bolt processes any number of input streams and produces any number of new output streams. Most of the logic of a computation goes into bolts, such as functions, filters, streaming joins, streaming aggregations, talking to databases, and so on.”

Page 14: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

/sin’fɒnjə/Module: Types

DRAIN?

Page 15: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

/sin’fɒnjə/

Define a Module

Load to Storm

Use in a Topology

Upload your Code

Share on Sinfonier

Module: Life Cycle

Page 16: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

/sin’fɒnjə/

Make a Topology

Run on Storm

Check Dashboard

Show results

Topology

Page 17: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

cat /var/log/named/query.log | grep "IN A" | awk '{ print $6 }' | awk -F"#" '{print $1}' |sort -n | uniq -c | sort -rn | head | awk '{ printf $1",";system("curl -s http://freegeoip.net/csv/"$2 | cut –d”,” –f3 )}’

curl --retry 3 --insecure -s https://www.rootedcon.es/ | grep -E 'href="http://.*rootedcon\.es'| awk -F"href=\"" '{print $2}' | sed 's|\".*||g' | xargs curl -s -o /dev/null --write-out "%{http_code}:%{size_download}\n"| awk -F":" '{ if ( $1 == "200") { print "RSS size: " $2} }'

crontab -l# m h dom mon dow command@reboot /usr/bin/python /home/charlie/.ave_phoenix.py30 7,15,23 * * * /home/charlie/vigila/gauchap.sh –tweet fotos 2>&1 >/dev/null

/sin’fɒnjə/ Shell Scripting

Page 18: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

/sin’fɒnjə/

Demo & Use cases

Page 19: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

/sin’fɒnjə/TweetMon

Page 20: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

/sin’fɒnjə/TorrentPeer

Page 21: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

/sin’fɒnjə/Crawler

Page 22: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

/sin’fɒnjə/Roadmap

Fun & Profit

Community

Page 23: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

/sin’fɒnjə/We Want You

Page 24: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]

/sin’fɒnjə/

Become a Beta Tester

http://sinfonier-project.net/

http://tinyurl.com/sinfonier