Transcript Control Spam by Use of Greylisting

Control Spam by the Use of
Greylisting
Torgny Hallenmark
LDC - Computing Center
Lund University, Sweden
[email protected]
TERENA Networking Conference 2005
[1]
2005-06-07
Lund University
Lund University, Sweden
• Located in the very south of Sweden
• One of the biggest universities in Scandinavia
• Has almost all faculties
• 40 000 students
• 6 000 employees
• www.lu.se for more info
[2]
Lund University
LU – employees and students
• Parallel mail servers used
• Same software used mainly
• Sun Java Enterprise System (JES)
• Sun JES Messaging Server
• Sun JES Directory Server (LDAP)
• Sendmail used in mail gateway
• Controlling systems: LUCAT and LADOK
[3]
Mail systems
LUCAT
LADOK
LDAP
LDAP
employees
students
mail server
mail server
Cassandra
[4]
Piraten
Mail systems
New mail control structure
• Central mail gateway for spam and virus
detection
• 3-4 parallel SunFire V240 servers
• In production since July 2004
• Spam detection: SpamAssassin, Greylisting
• Virus detection: Sophos, ClamAV
• Address verification: Only messages to valid
addresses on our domains are accepted
[5]
Mail systems
Spam and virus
detection
Argus1-3
mail.lu.se
mail.lu.se
mail.lu.se
mail server
local mail server
[6]
POP
IMAP
webmail
Mail systems
Mail servers -- central / local
• Central mail server with services for POP,
IMAP and webmail
• 75% of employees are using the central mail
service
• Local mail servers exist in some departments
• Local mail servers can use the central mail
gateway for spam and virus detection
[7]
E-mail: Virus detection
Virus detection in mail
• Software: Sophos and ClamAV
• Flagging in Subject: ***VIRUS***
• Virus infected attachments are removed, info
text is inserted
• Also possible to check for ”bad” file types
[8]
E-mail: Spam detection
Spam detection in mail
• Software: SpamAssassin
• Spam checks made, giving spam points
• Also RBL blacklists may give spam points
• Flagging in header: X-Spam-Flag
• Flagging in Subject: ***SPAM***
• No messages are trashed centrally, only
flagging is used (our policy)
• User must set up filter rules in his mail
program
[9]
Spam control: Greylisting
Spam control: Greylisting added
• Set in production on 1 July 2004
• Immediate impact !
• Spam is no longer a problem !!!
• 90-95% of earlier spam is just gone !
• Spam messages are not received, means
less messages to check (for both spam and
virus)
• No decision to take (if you would like to
thrash messages)
[10]
Spam control: Greylisting
Greylisting effect:
70
60
num ber
50
40
Total
Marked
30
20
10
day
Messages counted in a user mailbox
[11]
121
116
111
106
101
96
91
86
81
76
71
66
61
56
51
46
41
36
31
26
21
16
11
6
1
0
Spam control: Greylisting
Greylisting effect:
(Graph from Umeå University)
[12]
Spam control: Greylisting
Some user reactions:
• This is fantastic! Now you can again use e-mail like in
the old days!
• The spam is gone! How did you do this? Lots of
thanks!
• Really magic! Earlier I got 200 spam messages a
day, now I see at most two! Thank you for an
excellent work!
• It is almost sad with so few messages in my inbox …
• There must be something wrong with the mail
system, I hardly get any mail at all …
[13]
Spam control: Greylisting
Greylisting technique
•
•
•
•
Evan Harris: The Next Step in the Spam Control War
http://projects.puremagic.com/greylisting/
Using Internet SMTP standard (RFC 821)
The trick is following SMTP status:
451 4.7.1 TempFail – Please try again later
• Status code 451 can be handled by ”real” SMTP
mail servers (put message in queue, try to resend
it later)
• Spam spreading programs can not treat 451 status
info …
(not yet …)
[14]
Spam control: Greylisting
Greylisting technique
•
For incoming messages following ”triplet” is
checked:
1) IP address for sending SMTP server
2) Sender address (envelope sender)
3) Receiver address (envelope recipient)
•
•
[15]
If this triplet has not been seen earlier:
Send SMTP status 451, TempFail
If this triplet has been seen at least 5 minutes ago:
Accept the incoming message
Spam control: Greylisting
Greylisting technique
• A data base is needed (MySQL):






[16]
Time that triplet was first seen
Time that triplet blocking will expire
Time that triplet record itself will expire
Number of blocked delivery attempts
Number of messages sucessfully passed
Some other data
Spam control: Greylisting
Greylisting technique
• Some configuration parameters:
 Unknown triplet, initial delay
(default: 1 hour, our value: 5 minutes)
 Lifetime of new triplets that have not yet allowed a
mail to pass
(default: 5 hours, our value: 30 hours)
 Lifetime of auto-whitelisted triplets that have
allowed mail to pass
(default: 36 days)
[17]
Spam control: Greylisting
Greylisting – not always
• Manual whitelisting possible:
 Can be done for sending SMTP server, sender
address, receiver address
 Our own IP series are whitelisted (making the mail
gateway accept outgoing messages from our local
mail clients)
 Some ”odd” SMTP servers with problems with
Greylisting may be whitelisted. (But why not fix
those servers in stead?)
[18]
Spam control: Greylisting
Greylisting – any problems?
• Possible problems:
 First delivery is always delayed (for an unknown
triplet)
 Some mail servers are really not following Internet
SMTP standards, i e they don’t know how to
handle SMTP status 451
 Some mail servers have enormous spool queues,
making resend of messages something that might
happen first in a very distant future …
 Greylisting is no final solution to the spam
problem. Spammers will learn and adapt.
But don’t tell them … !?
[19]
Spam control: Greylisting
Greylisting software
 We use it together with Sendmail
 But Greylisting can work with others:
 Exim
 Qmail
 Qpsmtpd
 Postfix
 Squirrelmail
 Mail proxies
[20]
Spam control: Greylisting
Software used at Lund University





Sendmail 8.13
Greylisting (invoked via Sendmail Milter function)
MailScanner 4.31
SpamAssassin 2.63
Some RBLs (used from SpamAssassin)
 Sophos anti-virus
 ClamAV anti-virus
[21]
Spam control: Greylisting
More Greylisting information
http://projects.puremagic.com/greylisting/
[22]
Lund University e-mail policy
E-mail policy proposed
 Outgoing e-mail will be accepted only from a few
verified SMTP servers (very few servers running
spam and virus programs)
 All incoming e-mail must pass a central mail
gateway performing spam and virus checking
 Also internal e-mail (within the university) should
pass spam and virus checking
[24]
Lund University e-mail policy
Why use local mail servers?
 Goal: Reduce number of local mail servers
 Since earlier: Only certified SMTP servers are
accepted in the network (certified servers are
listed in routers). Only certified servers can
receive SMTP mail (via port 25).
 Certification is mainly an open relay check.
 Make central mail services better! No need for
local mail servers.
[25]