Differences

This shows you the differences between two versions of the page.

--- hints:classic_linux_mail_flow [2021/03/02 07:46]
admin [Run Procmail and SpamAssassin]
+++ hints:classic_linux_mail_flow [2021/04/03 07:41] (current)
admin [Receive a message from the network]
@@ Line 2: / Line 2: @@
 Mail received from the Internet flows through the system as described below.
+The below discussion is somewhat intricate. You don't need to know any of this to get the benefit of mail in the [[:Classic Linux]] environment. But if you are using a .forward file to forward mail to any off-site location (Gmail, Yahoo, etc.), this disussion will help you understand how and why some spam might get forwarded too.
 ===== Receive a message from the network =====
@@ Line 10: / Line 12: @@
 Postfix then feeds the same message to Rspamd in real time, i.e., while the same SMTP transaction is in progress.
-Rspamd does a large number of spam checks, including checking the headers added by OpenDKIM, and checking for DMARC and SPF records in DNS.
 Rspamd feeds the same message to Clamd, which checks for viruses and returns a result to Rspamd.
-Rspamd then returns its final findings to Postfix, which may ask Postfix to:
+Rspamd does a large number of spam checks, including checking the headers added by OpenDKIM, and checking for DMARC and SPF records in DNS.((For more information about how DKIM, SPF, and DMARC records work, see [[hints:DMARC DNS entry and mystery email reports]].))
+Rspamd then returns its final findings to Postfix. These findings may ask Postfix to:
-  * Accept the message as contains no spam and no virus.
   * Reject the message because it contains a virus.
-  * Reject the message because it contains spam with high certainty.
+  * Reject the message because it contains guaranteed spam.
-  * Greylist the message, because it appears to be spam but not with high certainty.
+  * Greylist the message because it contains likely spam.
+  * Accept the message because it contains no spam and no virus.
-Postfix acts on these findings. It will accept the message from the remote SMTP client, or it will send a permanent rejection reply to the remote SMTP client, or it will do greylisting by sending a temporary rejection to the remote SMTP client (which means try again later).((Most infected Microsoft Windows machines that are part of a botnet being used to send spam will not retry sending the same spam message, so greylisting eliminates almost all spam sent by such machines. The next time the same client sends a message from the same sender to the same recipient, within a predetermined period, it will not be greylisted.))
+Postfix acts on these findings. It will send a permanent rejection reply to the remote SMTP client due to a virus or guaranteed spam, or do greylisting by sending a temporary rejection to the remote SMTP client (which means try again later)((Most infected Microsoft Windows machines that are part of a botnet being used to send spam will not retry sending the same spam message, so greylisting eliminates almost all spam sent by such machines. The next time the same client sends a message from the same sender to the same recipient, within a predetermined period, it will not be greylisted.)), or accept the message from the remote SMTP client. Due to the high incidence of spam and viruses, only about 20% of incoming mail is unconditionally accepted. Most is rejected outright, and some is greylisted.
 If Postfix accepts the message, it expands aliases and domain names until it determines the final recipient, which in Classic Linux is always a Linux user. Let's call that user the first user, also called USER.
@@ Line 30: / Line 32: @@
 Postfix checks to see if the first user USER has a .forward file in their home directory. If the message was addressed using [[:plus addressing]], then Postfix will also first look for a .forward file whose name is of the form .forward-extension or .forward+extension where **-extension** or **+extension** is the extension part of the recipient address. If the message was addressed to USER-abc@rahul.net, Postfix will first look for a file called .forward-abc and then for a file called .forward. If the message was addressed to USER+xyz@example.com, Postfix will first look for a file called .forward+xyz and then for a file called .forward.
-Postfix will follow instructions in first such file that was found,if any. So the message might be forwarded to some other address, or fed to a program, or some combination of these.
+Postfix will follow instructions in first such file that was found, if any; and if so, the message might be forwarded to some other address, or fed to a program, or some combination of these.
 ===== Run Procmail and SpamAssassin =====
@@ Line 42: / Line 44: @@
 Taking into account these preferences, Procmail now does the final mail delivery. If the first user has a .procmailrc file, then instructions in it will be followed. Otherwise Procmail simply delivers the message into the first user's Maildir directory in [[:hints:maildir]] format, either into the user's inbox, or into the spam folder.
-Because spam-checking is done in sequence by Rspamd and SpamAssassin, you will often find two sets of headers in a message, some added by Rspamd and some by SpamAssassin. Rspamd adds headers only if it finds at least a little spam. SpamAssasin always adds headers, but makes them verbose only if it finds some spam, otherwise the SpamAssasin headers for not-spam are quite brief.
+Because spam-checking is done in sequence by Rspamd and SpamAssassin, you will often find two sets of headers in a message, some added by Rspamd and some by SpamAssassin. The verbosity of these headers may vary depending on the amount of spam found in a message.
 If the first user has a .forward file that sends mail elsewhere, Procmail is not run, and therefore SpamAssassin is not run.
@@ Line 48: / Line 50: @@
 If the first user's .forward file causes mail to go to a second user, then Postfix goes back to the step **check for .forward file** above, and looks for the second user's .forward file, if any. If the second user doesn't have a .forward file, then Procmail will run for the second user and Procmail will call SpamAssassin for the user.
-If the first user's .forward file causes mail to go off-site (i.e., mail is forwarded to some other place like Gmail, Yahoo, etc.), then SpamAssassin will never run for the first user. However, Rspamd already ran, so anything recognized as spam with high certainty has already been rejected. But likely spam, not recognized as spam with high certainty, will be forwarded.
+If the first user's .forward file causes mail to go off-site (i.e., mail is forwarded to some other place like Gmail, Yahoo, etc.), then SpamAssassin will never run for the first user. However, Rspamd already ran, so guaranteed spam — anything recognized as spam with high certainty — has already been rejected. But likely spam, not recognized as spam with high certainty, will be forwarded — see the discussion of spam thresholds below.
 ===== Spam thresholds =====
-The threshold for rejecting spam based on Rspamd's findings is kept somewhat high, so that we avoid false positives that would cause legitimate mail to be rejected and thus never reach the recipient. So although Rspamd effectively detects spam, it still allows quite a bit of spam to go through. When SpamAssassin runs, no mail is rejected outright, but merely put into a spam folder. This way we may misfile some legitimate mail as spam but it will not be lost.
+The threshold for rejecting spam based on Rspamd's findings is kept somewhat high, so that we avoid false positives that would cause legitimate mail to be rejected and thus never reach the recipient. So although Rspamd effectively detects essentially all spam, it blocks guaranteed spam but allows likely spam to go through, as a choice. When SpamAssassin runs, it will usually detect the remaining likely spam, and will cause Procmail to deliver the spam mail into the user's spam folder.
+So to summarize, the combination of Rspamd and SpamAssassin will catch almost all spam, the guaranteed spam being rejected by Rspamd when it first arrives, and the likely spam being caught by SpamAssassin and filed into the user's spam folder.
+Because of Rspamd's high threshold for rejecting spam, and because SpamAssassin does not run when a .forward file is used, mail forwarded with a .forward file will contain likely spam but not guaranteed spam.
-Because of Rspamd's high threshold for rejecting spam, and because SpamAssassin does not run when a .forward file is used, mail forwarded with a .forward file will contain some spam. You can make the amount of spam much lower by [[hints:example_procmailrc_file_forwarding|forwarding mail with a Procmail script]] and not with a .forward file.
+You can make the amount of forwarded spam quite a bit lower by [[hints:example_procmailrc_file_forwarding|forwarding mail with a Procmail script]] and not with a .forward file.

A2I Communications Info Site

User Tools

Site Tools

Differences

Page Tools