User Tools

Site Tools


hints:classic_linux_mail_flow

Classic Linux mail flow

Mail received from the Internet flows through the system as described below.

The below discussion is somewhat intricate. You don't need to know any of this to get the benefit of mail in the Classic Linux environment. But if you are using a .forward file to forward mail to any off-site location (Gmail, Yahoo, etc.), this disussion will help you understand how and why some spam might get forwarded too.

Receive a message from the network

An email message is received from a remote SMTP client by the mail transport agent Postfix.

Postfix feeds the message to the software program OpenDKIM in real time, i.e., while the SMTP transaction is in progress. OpenDKIM checks for DKIM records in DNS and also checks for any DKIM digital signature in the message, and adds headers to the message containing the result of these checks.

Postfix then feeds the same message to Rspamd in real time, i.e., while the same SMTP transaction is in progress.

Rspamd feeds the same message to Clamd, which checks for viruses and returns a result to Rspamd.

Rspamd does a large number of spam checks, including checking the headers added by OpenDKIM, and checking for DMARC and SPF records in DNS.1)

Rspamd then returns its final findings to Postfix. These findings may ask Postfix to:

  • Reject the message because it contains a virus.
  • Reject the message because it contains guaranteed spam.
  • Greylist the message because it contains likely spam.
  • Accept the message because it contains no spam and no virus.

Postfix acts on these findings. It will send a permanent rejection reply to the remote SMTP client due to a virus or guaranteed spam, or do greylisting by sending a temporary rejection to the remote SMTP client (which means try again later)2), or accept the message from the remote SMTP client. Due to the high incidence of spam and viruses, only about 20% of incoming mail is unconditionally accepted. Most is rejected outright, and some is greylisted.

If Postfix accepts the message, it expands aliases and domain names until it determines the final recipient, which in Classic Linux is always a Linux user. Let's call that user the first user, also called USER.

Check for .forward file

Postfix checks to see if the first user USER has a .forward file in their home directory. If the message was addressed using plus addressing, then Postfix will also first look for a .forward file whose name is of the form .forward-extension or .forward+extension where -extension or +extension is the extension part of the recipient address. If the message was addressed to USER-abc@rahul.net, Postfix will first look for a file called .forward-abc and then for a file called .forward. If the message was addressed to USER+xyz@example.com, Postfix will first look for a file called .forward+xyz and then for a file called .forward.

Postfix will follow instructions in first such file that was found, if any; and if so, the message might be forwarded to some other address, or fed to a program, or some combination of these.

Run Procmail and SpamAssassin

If no .forward (or .forward-extension or .forward+extension) file was found, Postfix now runs the Procmail program, and feeds it the message.

At this point Postfix's role for this message is over, and Procmail has taken over.

Procmail now feeds the message to SpamAssassin, which checks the message for spam and returns a status back to Procmail. Procmail also consults the first user's preferences as to whether or not spam as determined by SpamAssassin should go into the first user's normal inbox or into a spam folder, and whether spam above a certain level should be discarded, and follows these preferences.

Taking into account these preferences, Procmail now does the final mail delivery. If the first user has a .procmailrc file, then instructions in it will be followed. Otherwise Procmail simply delivers the message into the first user's Maildir directory in maildir format, either into the user's inbox, or into the spam folder.

Because spam-checking is done in sequence by Rspamd and SpamAssassin, you will often find two sets of headers in a message, some added by Rspamd and some by SpamAssassin. The verbosity of these headers may vary depending on the amount of spam found in a message.

If the first user has a .forward file that sends mail elsewhere, Procmail is not run, and therefore SpamAssassin is not run.

If the first user's .forward file causes mail to go to a second user, then Postfix goes back to the step check for .forward file above, and looks for the second user's .forward file, if any. If the second user doesn't have a .forward file, then Procmail will run for the second user and Procmail will call SpamAssassin for the user.

If the first user's .forward file causes mail to go off-site (i.e., mail is forwarded to some other place like Gmail, Yahoo, etc.), then SpamAssassin will never run for the first user. However, Rspamd already ran, so guaranteed spam — anything recognized as spam with high certainty — has already been rejected. But likely spam, not recognized as spam with high certainty, will be forwarded — see the discussion of spam thresholds below.

Spam thresholds

The threshold for rejecting spam based on Rspamd's findings is kept somewhat high, so that we avoid false positives that would cause legitimate mail to be rejected and thus never reach the recipient. So although Rspamd effectively detects essentially all spam, it blocks guaranteed spam but allows likely spam to go through, as a choice. When SpamAssassin runs, it will usually detect the remaining likely spam, and will cause Procmail to deliver the spam mail into the user's spam folder.

So to summarize, the combination of Rspamd and SpamAssassin will catch almost all spam, the guaranteed spam being rejected by Rspamd when it first arrives, and the likely spam being caught by SpamAssassin and filed into the user's spam folder.

Because of Rspamd's high threshold for rejecting spam, and because SpamAssassin does not run when a .forward file is used, mail forwarded with a .forward file will contain likely spam but not guaranteed spam.

You can make the amount of forwarded spam quite a bit lower by forwarding mail with a Procmail script and not with a .forward file.

1)
For more information about how DKIM, SPF, and DMARC records work, see DMARC DNS entry and mystery email reports.
2)
Most infected Microsoft Windows machines that are part of a botnet being used to send spam will not retry sending the same spam message, so greylisting eliminates almost all spam sent by such machines. The next time the same client sends a message from the same sender to the same recipient, within a predetermined period, it will not be greylisted.
hints/classic_linux_mail_flow.txt · Last modified: 2021/04/03 07:41 by admin