Transcript PowerPoint Presentation - New Mail Filtering Features in

Anti-Spam & Anti-Virus
WiscMail Implementation
University of Wisconsin - Madison
CSG Workshop
September 21, 2004
Message Composition - Fall 2004
The Spam Threat
•
Users don’t want spam
–
–
–
•
Lost productivity
Offensive, Embarrassing
Legitimate messages get lost in the sea of spam
Spam isn’t going away
–
–
–
People buy from spammers
Legislation has not been effective
The SMTP protocol is inadequate
o
•
It allows spammers to forge message information
Spam is difficult to detect
–
–
Spammers learn how to get past filters
Legitimate messages WILL be lost
The Spam Threat
•
Anti-Spam is difficult to support
–
–
–
•
Users don’t like misclassifications
Client based anti-spam solutions interfere
Authorized mass-mailers want special treatment
Spammers use malware
–
–
–
–
Viruses “spam” themselves in mass quantities
Disinfected virus message clog Inboxes
Compromised computers DoS attack Anti-Spam services (RBLs)
Compromised computers send spam from inside the network
Anti-Spam Project Goals
•
Reduce spam by 80% from current levels
•
Users must be able to receive spam if they want (Opt-Out)
•
Provide an option to select levels of filtering
•
System must perform well and be scalable as message volumes
increase
•
Provide a Web Interface to system
•
Compatible with existing infrastructure
•
Vendor supported system
Anti-Virus & Anti-Spam Integration
•
Why integrate anti-spam and anti-virus?
–
Faster processing
o
Messages are only opened once
–
Server consolidation
–
Virus messages can be treated as spam
o
Keeps the clutter out of the Inbox
How it works
1.
Scan all incoming messages for spam and viruses
–
–
–
2.
All potentially unsafe messages are scanned
Messages are marked with a spam “score” and then delivered as
intended
Virus messages are deleted or disinfected
Filter the messages
–
–
Users choose whether or not to filter spam messages
Users choose what threshold (based on spam score) to filter spam
Spam Scanning
•
Allow mail from trusted sources to pass unaffected
•
All other mail is marked in the headers
–
–
e.g. X-Spam-Score: ****
7 score levels
o
o
•
0 asterisks means the message is likely not spam
7 asterisks means the message is likely spam
Deliver all messages to recipient
Virus Scanning
•
Message is infected with a “junk” virus
–
–
e.g. netsky, bagel, mydoom, …
Delete messages without notification to sender or recipient
o
•
Message is infected with a virus
–
–
–
•
The induced message load from outbreaks causes delays for legitimate mail
Remove virus
Mark message as spam
Append [VIRUS] to subject
Message contains a suspicious attachment (exe, pif, scr, …)
–
–
Do nothing unless there is an outbreak
During an outbreak, treat these messages like viruses
Spam Filtering
•
Server-side filtering service
–
–
–
•
Custom built interface that allows users to configure individual filters to
move messages into IMAP folders
Based on Sieve RFC
Compatible with IMAP and Web Mail users
‘Junk Mail’ Folder
–
–
Reserved IMAP folder
Mail in ‘Junk Mail’ is deleted after 15 days of age
Spam Filtering
•
Junk Mail Filter
–
–
•
Accept List Filter
–
•
Moves all mail from specified senders to the ‘Junk Mail’ folder
Mailing Lists Filter
–
•
Keeps all mail from specified senders in the Inbox
Block List Filter
–
•
Users specify desired spam threshold (based on spam score)
Moves all spam marked at the specified level (or higher) into ‘Junk Mail’ folder
Keeps mail addressed to list addresses in the Inbox
Custom Filters
–
–
Users can create filters to move messages into IMAP folders
e.g. “If the Subject contains ‘CSG’ move the message into the CSG folder”
Spam Filtering - Issues
•
POP users see “disappearing email”
–
–
–
•
Mail is “POPed” from the Inbox only
POP users have to use Web Mail to see filtered mail
Alternatively, client-side filters can be used in conjunction with marked
spam messages
Conflicts with client-based anti-spam filters
–
More misclassifications
o
–
Support confusion
o
–
Client filters are looking for spam that isn’t there
Users see two “junk” folders
Most new email clients have spam filtering enabled by default
o
Outlook 2003, Eudora 6, Mozilla variants
How Mail is Treated
Server
Filters
Incoming Mail from
WiscMail Users
X
All Other Incoming
Mail
X
Outgoing Mail
X
Spam
Scan
X
Virus
Scan
User
Filters
X
X
X
X
X
N/A
Other Tools & Techniques
•
Server Filters
–
–
–
–
•
Similar to user-level filters, but applies to all messages
Saves load on spam and virus scanners, by deleting or rejecting at
the front door
Hundreds of thousands of SoBig messages stopped during 2003
outbreak
Only works if the messages have definable characteristics
Site RBL
–
–
–
Real-time Blocking List
DNS Based
Allows us to dynamically block abusive computers from connecting
to our mail servers
Other Tools & Techniques
•
Require SMTP Authentication
–
•
Compromised (zombie) machines are becoming the major source
of spam
Rate Limit incoming and outgoing traffic
–
Limit abuse from spammers
Traffic Patterns
Traffic Patterns - Virus
Sophos PureMessage
•
Direct integration with our mail software (Sun iMS)
•
•
Uses multiple spam detection technologies
–
•
Allows for specific actions based on message characteristics
Many message actions provided
–
•
Heuristics, RBLs, checksums
Customizable site policy based on Sieve RFC
–
•
Also supports Sendmail and Postfix
Header/body modification, quarantining, discard, drop/replace
attachments
Server cluster management
Sophos PureMessage
•
Honey potting
•
Dummy accounts set up to collect spam
•
Misclassification submittal process
•
Hourly automatic anti-spam heuristic updates
•
Hourly automatic anti-virus IDE file updates
•
Integration with anti-virus
•
End-user quarantine management
•
Not in use by UW-Madison
New Technologies
•
Sender Authentication by IP Addresses
–
What is it?
o
o
o
–
SPF
o
o
–
Helps prevent address spoofing
Allows administrators to specify the computers that are authorized to
use addresses in a particular domain
Stores information in DNS
Open project (http://spf.pobox.com/)
16% of mail domains have published SPF records
SenderID
o
o
Microsoft’s implementation, formerly called CallerID
proposal is having a hard time getting approved by the IETF
o
–
Relies on proprietary technology
The technology is still in too much flux to be fully embraced
New Technologies
•
Sender Authentication with Content Signing
–
What is it?
o
o
–
DomainKeys
o
o
–
Helps prevent address spoofing
Uses SSL certificates to ensure that messages are sent by legitimate
senders from the domain
Specification submitted to IETF by Yahoo
Stores certificates in DNS
This technology is not as advanced as sender authentication with
IP addresses
Anti-Spam Technical Alliance
•
Anti-Spam Technical Alliance
–
•
Yahoo!, Microsoft, EarthLink and AOL
Recommendations to Help Stop Spam
–
http://docs.yahoo.com/docs/pr/release1169.html
–
Address email address forgery with sender authentication
Recommendations for ISPs
–
•
–
e.g. rate limiting, limit port 25, close open relays, shut down zombie
spammers
Recommendations for consumers
•
e.g. install firewalls and desktop a/v, make use of spam filtering
technologies that are provided by their ISP
Future Plans
•
Sender Authentication (SPF)
–
–
•
Publish SPF records
Filter based on SPF
Possible use of quarantining
–
Advantages
o
o
o
–
Keep spam on spam servers instead of Junk Mail folder
Users can choose what to do with the messages that are quarantined
Users can correct the spam server so that it makes the right decisions
in the future
Disadvantages
o
o
There are compatibility issues with our infrastructure
Users would have to learn yet another process
Question and Answer