Anti-Spam & Anti-Virus WiscMail Implementation

University of Wisconsin - Madison

CSG Workshop

September 21, 2004

Message Composition - Fall 2004

The Spam Threat

• Users don’t want spam– Lost productivity– Offensive, Embarrassing– Legitimate messages get lost in the sea of spam

• Spam isn’t going away– People buy from spammers– Legislation has not been effective– The SMTP protocol is inadequate

o It allows spammers to forge message information

• Spam is difficult to detect– Spammers learn how to get past filters– Legitimate messages WILL be lost

The Spam Threat

• Anti-Spam is difficult to support– Users don’t like misclassifications– Client based anti-spam solutions interfere– Authorized mass-mailers want special treatment

• Spammers use malware– Viruses “spam” themselves in mass quantities– Disinfected virus message clog Inboxes– Compromised computers DoS attack Anti-Spam services (RBLs)– Compromised computers send spam from inside the network

Anti-Spam Project Goals

• Reduce spam by 80% from current levels

• Users must be able to receive spam if they want (Opt-Out)

• Provide an option to select levels of filtering

• System must perform well and be scalable as message volumes increase

• Provide a Web Interface to system

• Compatible with existing infrastructure

• Vendor supported system

Anti-Virus & Anti-Spam Integration

• Why integrate anti-spam and anti-virus?

– Faster processingo Messages are only opened once

– Server consolidation

– Virus messages can be treated as spamo Keeps the clutter out of the Inbox

How it works

1. Scan all incoming messages for spam and viruses– All potentially unsafe messages are scanned– Messages are marked with a spam “score” and then delivered as

intended– Virus messages are deleted or disinfected

2. Filter the messages– Users choose whether or not to filter spam messages– Users choose what threshold (based on spam score) to filter spam

Spam Scanning

• Allow mail from trusted sources to pass unaffected

• All other mail is marked in the headers– e.g. X-Spam-Score: ****– 7 score levels

o 0 asterisks means the message is likely not spamo 7 asterisks means the message is likely spam

• Deliver all messages to recipient

Virus Scanning

• Message is infected with a “junk” virus– e.g. netsky, bagel, mydoom, …– Delete messages without notification to sender or recipient

o The induced message load from outbreaks causes delays for legitimate mail

• Message is infected with a virus– Remove virus– Mark message as spam– Append [VIRUS] to subject

• Message contains a suspicious attachment (exe, pif, scr, …)– Do nothing unless there is an outbreak– During an outbreak, treat these messages like viruses

Spam Filtering

• Server-side filtering service– Custom built interface that allows users to configure individual filters to

move messages into IMAP folders– Based on Sieve RFC– Compatible with IMAP and Web Mail users

• ‘Junk Mail’ Folder– Reserved IMAP folder– Mail in ‘Junk Mail’ is deleted after 15 days of age

Spam Filtering

• Junk Mail Filter– Users specify desired spam threshold (based on spam score)– Moves all spam marked at the specified level (or higher) into ‘Junk Mail’ folder

• Accept List Filter– Keeps all mail from specified senders in the Inbox

• Block List Filter– Moves all mail from specified senders to the ‘Junk Mail’ folder

• Mailing Lists Filter– Keeps mail addressed to list addresses in the Inbox

• Custom Filters– Users can create filters to move messages into IMAP folders– e.g. “If the Subject contains ‘CSG’ move the message into the CSG folder”

Spam Filtering - Issues

• POP users see “disappearing email”– Mail is “POPed” from the Inbox only– POP users have to use Web Mail to see filtered mail– Alternatively, client-side filters can be used in conjunction with marked

spam messages

• Conflicts with client-based anti-spam filters– More misclassifications

o Client filters are looking for spam that isn’t there– Support confusion

o Users see two “junk” folders– Most new email clients have spam filtering enabled by default

o Outlook 2003, Eudora 6, Mozilla variants

How Mail is Treated

Server Filters

Spam Scan

Virus Scan

User Filters

Incoming Mail from WiscMail Users

X X X

All Other Incoming Mail

X X X X

Outgoing Mail X X N/A

Other Tools & Techniques

• Server Filters– Similar to user-level filters, but applies to all messages– Saves load on spam and virus scanners, by deleting or rejecting at

the front door– Hundreds of thousands of SoBig messages stopped during 2003

outbreak– Only works if the messages have definable characteristics

• Site RBL– Real-time Blocking List– DNS Based– Allows us to dynamically block abusive computers from connecting

to our mail servers

Other Tools & Techniques

• Require SMTP Authentication– Compromised (zombie) machines are becoming the major source

of spam

• Rate Limit incoming and outgoing traffic– Limit abuse from spammers

Traffic Patterns

Traffic Patterns - Virus

Sophos PureMessage

• Direct integration with our mail software (Sun iMS)• Also supports Sendmail and Postfix

• Uses multiple spam detection technologies– Heuristics, RBLs, checksums

• Customizable site policy based on Sieve RFC– Allows for specific actions based on message characteristics

• Many message actions provided– Header/body modification, quarantining, discard, drop/replace attachments

• Server cluster management

Sophos PureMessage

• Honey potting• Dummy accounts set up to collect spam

• Misclassification submittal process

• Hourly automatic anti-spam heuristic updates

• Hourly automatic anti-virus IDE file updates

• Integration with anti-virus

• End-user quarantine management• Not in use by UW-Madison

New Technologies

• Sender Authentication by IP Addresses– What is it?

o Helps prevent address spoofingo Allows administrators to specify the computers that are authorized to use

addresses in a particular domaino Stores information in DNS

– SPFo Open project (http://spf.pobox.com/)o 16% of mail domains have published SPF records

– SenderIDo Microsoft’s implementation, formerly called CallerIDo proposal is having a hard time getting approved by the IETF

o Relies on proprietary technology

– The technology is still in too much flux to be fully embraced

New Technologies

• Sender Authentication with Content Signing– What is it?

o Helps prevent address spoofingo Uses SSL certificates to ensure that messages are sent by legitimate

senders from the domain– DomainKeys

o Specification submitted to IETF by Yahooo Stores certificates in DNS

– This technology is not as advanced as sender authentication with IP addresses

Anti-Spam Technical Alliance

• Anti-Spam Technical Alliance– Yahoo!, Microsoft, EarthLink and AOL

• Recommendations to Help Stop Spam– http://docs.yahoo.com/docs/pr/release1169.html– Address email address forgery with sender authentication– Recommendations for ISPs

• e.g. rate limiting, limit port 25, close open relays, shut down zombie spammers

– Recommendations for consumers• e.g. install firewalls and desktop a/v, make use of spam filtering

technologies that are provided by their ISP

Future Plans

• Sender Authentication (SPF)– Publish SPF records– Filter based on SPF

• Possible use of quarantining– Advantages

o Keep spam on spam servers instead of Junk Mail foldero Users can choose what to do with the messages that are quarantinedo Users can correct the spam server so that it makes the right decisions

in the future– Disadvantages

o There are compatibility issues with our infrastructureo Users would have to learn yet another process

Question and Answer