Presented by

S.C. Leung

Log InfrastructureLog Infrastructure

• Time Synchronization of log servers

• Log relaying & centralization

• Log secure transport, Log integrity

• Log rotate

• Log reporting

• Log archive & storage

• Log server protection

• Time Sync is vital in correlation of log– Same Reference clock provides sequencing of events– Log servers sync to a timer server, which sync to an external time

server

• Network Time Protocol (NTP)– Run on UDP/123 (your firewall need to open in both directions)

• NTP v.3: RFC 1305 (symmetric key encryption); • NTP v.4 in dev (symmetric key and public key encryption)• Simple NTP (SNTP) v.4 RFC 2030 (e.g. w32time)

– Out of sync < 1 hour � delta increment– Out of sync > 1 hour � no change; should use ntpupdate to hard set

current time

• Time server hierarchy:– NTP client usually connected to stratum 2 or 3 servers– Stratum 2 servers: connected to stratum 1 servers directly– Stratum 1 servers: connected to external source of Universal Coordinated Time (UTC),

e.g. radio clock• The Hong Kong Observatory Network Time Service (www.hko.gov.hk/nts/ntime.htm)

– HKO time server: stdtime.gov.hk

• Other approaches: – HTP (www.clevervest.com/htp/htp.html) when firewall blocks NTP; not a protocol– GPS

Stratum 1

clock.cuhk.edu.hkLocation: The Chinese University of Hong Kong.Geographic Coordinates: 22:25:10N,114:12:22ESynchronization: NTP V3 Primary (TSS-100 GPS clock)Service Area: HK, TWN, China, and South East Asia Access Policy: open access

Stratum 2

ntp.cuhk.edu.hkLocation: The Chinese University of Hong Kong.Geographic Coordinates: 22:25:10N, 114:12:22ESynchronization: NTP V3 secondary (stratum 2), i686/LinuxService Area: HK, TWN, China, and South East AsiaAccess Policy: open access

• ntpd.conf • Windows sntp client

Serverpeer ntp1.cityu.edu.hkserver stdtime.gov.hkrestrict default notrust nomodifyrestrict 132.249.0.0 mask

255.255.0.0 nomodify

C:\>net time /setsntp:192.168.100.250C:\>net stop w32timeC:\>net start w32time

C:\>net time /querysntpThe current SNTP value is: 192.168.100.250

Clientspeer ntp1.cityu.edu.hkrestrict default notrust nomodifyrestrict 132.249.0.0 mask

255.255.0.0 nomodify

• What is syslog?– Developed by UC Berkeley as

part of sendmail

• Why Syslog?– Widely used in UNIX and Cisco– Can log to console, local

harddisk or remote host

• UDP/514

• Facilities & Severity Levels

Facility

authauthprivcrondaemon ftp kern lprmail news sysloguser uucplocal0…local7

Levels

0 Emerg1 Alert 2 Crit3 Err 4 Warning 5 Notice 6 Info7 Debug

• Syslog Daemon - syslogd

• Syslog Configuration– /etc/syslog.conf controls how much data is recorded, and

what becomes of it

• Log repository, e.g. – /dev/console– /var/adm/message– @192.168.1.100 # remote loghost

• syslog.conf format:Facility.level <Tab> action

– Facility indicate what’s sending the message

– Level indicate what the criticality of the message

# Log all kernel messages to the console.# Logging much else clutters up the screen.#kern.*[Tab]/dev/console#logging auth messages to centralized logging serversauth.info[Tab]@loghost.company.com

# Log anything (except mail) of level info or higher.# Don't log private authentication messages!*.info;mail.none;authpriv.none;cron.none[Tab]/var/log/messages

# The authpriv file has restricted access.authpriv.*[Tab]/var/log/secure

# Log all the mail messages in one place.mail.*[Tab]/var/log/maillog

# Log cron stuffcron.*[Tab]/var/log/cron

# Everybody gets emergency messages, plus log them on another machine.*.emerg

• No standard message formats, but usually something like

date:time Host/IP Service [Message]

Example: Sambasmb.conf

http://www.oreilly.com/catalog/samba/chapter/book/ch04_08.html

Syslog Priority Conversion

LOG_DEBUG4 and above

LOG_INFO3

LOG_NOTICE2

LOG_WARNING1

LOG_ERR0

Syslog PriorityLog Level

[global]log level = 3 ; log message of level 3 and lowersyslog = 1 ; log with level 1 or above but smaller

; than 3 sent to syslog server

---------------------------------------------------------[global]log level = 3 ;syslog only = yes ; log to syslog server for all messages of

; level 3 or lower

• Why centralize?– Central place to keep and

backup

– Provide trail when compromised

– Correlation & Analysis

– Intrusion detection

• Why not?– Single point of failure, if

only one log server

– Can be sniffed in transit

– Syslog UDP packets• no guarantee to arrive

• can be spoofed

– Mixed logs is hard to read

• Syslog Servers– UNIX Syslog– Windows Syslog (www.winsyslog.com) - commercial

• Windows to Syslog– NTSyslog (command line only)

• ntsyslog.sourceforge.net– Backlog

• www.intersectalliance.com/projects

– Triaction Syslog (commercial)• www.triaction.nl

– EventReporter with filter (commercial)• www.eventreporter.com

• Separate file systems for OS, binary and data• Run syslog and ssh only

– Remove all Inet & RPC services, disable unnecessary accounts

• (very seriously) recompile syslog to hide syslog.conf• http://www.asociacion-aecsi.es/doc/HoneyPot/Building_a_HoneyNet.pdf

• Run syslog –rat syslog server to receive remote logs

• Consider use a syslog replacement to solve problems

(*) Write down all log management procedure – useful for legal followup

• Turn on Cisco syslog

• Windows: EventReporter

• Firewall

• Samba

• BIND, etc.

Ref: [Tina Bird 2002]

• Single central host

• Relaying

File Svr Router Intranet

CentralLoghost

Branch1Loghost

Branch2Loghost

WWW Mail DNS

DMZLoghost

• Network Filtering– Use firewall to restrict syslog traffic from authorized only

sources• Encryption

– VPN tunnel• Integrity

– Use Syslog replacement with integrity• Authentication

– Syslog Authentication Protocol • Submitted for consideration as a IETF standard (Jul-2005)• http://www.ietf.org/internet-drafts/draft-ietf-syslog-sign-17.txt

(Nov-2005)

• Why replacement? – Fix [Tab] problem

– Security Enhancement

– Database support

– Compatibility and format conversion

• Candidates– Syslog-ng (windows as well, SHA1)

• http://www.balabit.com/products/syslog_ng/

– Modular Syslog• http://www1.corest.com/products/corewisdom/CW01.php

• Replace Syslogd and Klogd• Modular design � new feature can be added• Supports TCP (& UDP, UNIX named pipes and plain text)

• Integrity Check• Support Encryption (with two ends msyslog)• Support MySQL and PostgreSQL• Support relaying• Support filtering

– http://oss.coresecurity.com/projects/msyslog.html

• Mutliple platforms (UNIX, Linux, Solaris, AIX, MacOS, *BSD)– http://sourceforge.net/projects/msyslog/

• Protect loghost from attacker• Case: Centralize multiple web server log in DMZ• Remark: log host still vulnerable to DoS

Ref: http://www.giac.org/certified_professionals/practicals/gsec/3428.php

RealLoghost

no IPmac aa:bb:cc:dd:ee:ff

# tell the world such IP map to my LAN card MAC“arpspoof” 10.1.1.100 aa:bb:cc:dd:ee:ff

# snort rule: read udp/514 and log to ./snort/dmzloglog udp 10.1.1.0/24 any ->10.1.1.100/24 514 (logto: “dmzlog";)

www www www

PhantomLoghost

Web Server Farm

10.1.1.100

DMZ: 10.1.1.0/24

# forward all logs to phantom loghost

*.* @10.1.1.100

• Modify /etc/logrotate.d/syslog• logrotate is controlled through cron: called daily by /etc/cron.daily/logrotate.

# see "man logrotate" for details# rotate log files weekly (default)weekly

# create new (empty) log files after rotating old onescreate

/var/log/message {dailyrotate 90postrotate

/usr/bin/killall –HUP syslogdendscript

}

/var/log/security {rotate 12postrotate

/usr/bin/killall –HUP syslogdendscript

}

– Event View in another computer require the DLL file to understand the unique application and system event

– (Q165959 ) Reading a File Saved with the Event Viewer of Another Computer• Error message: “The description for Event ID (xxx) in Source (aservice) could not be found.

It contains the following insertion string(s): ... “• For application event logs messages DLLs

– HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\ Application• For system event logs messages DLLs

– HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\ System

• http://support.microsoft.com/kb/q165959/

• Create a new serialized log file everyday

• Integrity – signing

• Windows event log– backup the event dll files in %windir%\system32

• Backup to write-once media

• Remove log file from touch of hackers (Log to remote host, take backup off-site)

• Have a robot watching it and alert you– Panic

– Denied

– ssh failure 5 times

• Freeware Linux tool for automated log monitoring (use Perl to do tail –f)• Download: previously ftp://ftp.stanford.edu/general/security-tools/swatch/

swatch --config-file=/home/sc/swatch.conf --examine =/var/log/messagesswatch --config-file=/home/sc/swatch_apache.conf ---tail-file =/var/log/apache/messages

Ignore /news|CROND/

watchfor /[dD]enied|/DEN.*ED/echo boldbell 3mail=alert@pisa.org.hk, subject=Log_Denialexec "/etc/call_pager 5551234 08“

watchfor /router/bell mail=alert@pisa.org.hk, subject=Log_Routerexec

Ref http://www.spitzner.net/swatch.htmlhttp://www.linuxsecurity.com/index2.php?option=com_content&do_pdf=1&id=117281

command line (can run multiple swatch, one for each log file)

swatch.conf

• Event correlation rule types – if line-A

– if line-A more N times within a window of X sec.

– if line-A and line-B

– if line-A and not line-B

SEC• http://kodu.neti.ee/%7Eristo/sec/

• http://sixshooter.v6.thrupoint.net/SEC-examples/article.html

LogSurfer• http://www.cert.dfn.de/eng/logsurf/index-print.html

• Universal Reference for logging– http://www.loganalysis.org/

• Syslog Attack Signature (2002 Tina Bird)– http://www.seclib.com/seclib/ids.general/syslog-attack-sigs.pdf

• Building a Logging Infrastructure (Abe Singer)– http://www.secureitconf.com/OLD/2005/presentations/ID172-

LOGANALY.PPT

• Centralized Log Host– http://www.campin.net/newlogcheck.html

• Centralized logging using Logsentry in a large UNIX environment

– http://www.giac.org/certified_professionals/practicals/gsec/2256.php