T M D A

  
TMDA Homepage TMDA @ SourceForge      
Overview
Introduction
History
Features
Results & Testimonials
TMDA In Use
Press Coverage
 
Usage
Requirements
Download
Installation
Configuration
HOWTOs
 
Configuration
Overview
Pre-Configuration
Server Configuration
Client Configuration
Filter Specification
Filter Sources
 
Support
Troubleshooting
FAQ
Bugs & Patches
Mailing Lists
List Archive
External Resources
 
Author
Jason R. Mastaler
 
© 2001-2002
  

TMDA Filter Sources

In the following list of sources, the expected match field is documented as well as any optional or required arguments. Square brackets ([]) indicate the the argument is optional. Words in chevrons (<>) should be replaced by the appropriate option, without the chevrons.

This group of sources may be used in either incoming or outgoing filter files.

from <email_address>
to <email_address>
The from and to sources expect a match field of either an explicit email address or a wildcarded email address. The format of the email address is documented below.

The from source examines the envelope sender, any addresses in the From header field (not the UNIX mbox separator From_) and any addresses in the Reply-To header field in order to find a match.

The to source examines the envelope recipient but not the To header field, which is often bogus in spam email.

from-file [ -autocdb | -autodbm ] [ -optional ] <textfile>
to-file [ -autocdb | -autodbm ] [ -optional ] <textfile>
The from-file and to-file sources expect the name of a textfile as the match field. You can specify the entire path explicitly or use a leading '~' to represent the user's home directory, like the shell does. The match field is always the name of the textfile. You do not need to add '.cdb' or '.db' if you use the -auto* flags. It will be automatically appended to the filename. The format of the textfile is documented below.

Both from-file and to-file can take one of the -autocdb or -autodbm flags. The -autocdb and -autodbm flags are documented below.

If the -optional flag is given, the non-existence of the file is not an error. If the file should exist, don't specify this flag; the parser will log an error and will defer the mail so that you have a chance to fix the problem.

The from-file source examines the envelope sender, any addresses in the From header field (not the UNIX mbox separator From_) and any addresses in the Reply-To header field in order to find a match.

The to-file source examines the envelope recipient but not the To header field, which is often bogus in spam email.

from-cdb [ -optional ] <database.cdb>
to-cdb [ -optional ] <database.cdb>
The from-cdb and to-cdb sources expect a match field of a CDB database filename. You can specify the entire path or use a leading '~' to represent the user's home directory. You should specify the .cdb extension as part of the filename. The CDB format and expected contents are documented below.

If the -optional flag is given, the non-existence of the file is not an error. If the file should exist, don't specify this flag; the parser will log an error and will defer the mail so that you have a chance to fix the problem.

The from-cdb source examines the envelope sender, any addresses in the From header field (not the UNIX mbox separator From_) and any addresses in the Reply-To header field in order to find a match.

The to-cdb source examines the envelope recipient but not the To header field, which is often bogus in spam email.

from-dbm [ -optional ] <database.db>
to-dbm [ -optional ] <database.db>
The from-dbm and to-dbm sources expect the name of a DBM database in the match field. You can specify the entire path or use a leading '~' to represent the user's home directory. You should specify the .db extension as part of the filename. The DBM format and expected contents are documented below.

If the -optional flag is given, the non-existence of the file is not an error. If the file should exist, don't specify this flag; the parser will log an error and will defer the mail so that you have a chance to fix the problem.

The from-dbm source examines the envelope sender, any addresses in the From header field (not the UNIX mbox separator From_) and any addresses in the Reply-To header field in order to find a match.

The to-dbm source examines the envelope recipient but not the To header field, which is often bogus in spam email.

from-ezmlm [ -optional ] <path_to_subscribers_parent_dir>
to-ezmlm [ -optional ] <path_to_subscribers_parent_dir>
The from-ezmlm and to-ezmlm sources match against the subscriber list of an ezmlm mailing list. They expect the match field to be the full path of the parent directory of an ezmlm `subscribers' directory. You should not include the `subscribers' portion of the path.

If the -optional flag is given, the non-existence of the file is not an error. If the file should exist, don't specify this flag; the parser will log an error and will defer the mail so that you have a chance to fix the problem.

The from-ezmlm source examines the envelope sender, any addresses in the From header field (not the UNIX mbox separator From_) and any addresses in the Reply-To header field in order to find a match.

The to-ezmlm source examines the envelope recipient but not the To header field, which is often bogus in spam email.

from-mailman -attr=<attribute> [ -optional ] <path_to_list_dir>
to-mailman -attr=<attribute> [ -optional ] <path_to_list_dir>
The from-mailman and to-mailman sources match against addresses contained in a Mailman configuration database. The match field should be the full path to the list directory. Both Mailman 2.0 and 2.1-style configuration databases are supported.

The -mailman sources require you to specify an `attribute' to search. Use the -attr= flag to specify the name of an attribute contained in the database. For example, `members' (subscriber addresses), `digest_members' (digest subscriber addresses), or `owner' (list owner's address).

If the -optional flag is given, the non-existence of the file is not an error. If the file should exist, don't specify this flag; the parser will log an error and will defer the mail so that you have a chance to fix the problem.

The from-mailman source examines the envelope sender, any addresses in the From header field (not the UNIX mbox separator From_) and any addresses in the Reply-To header field in order to find a match.

The to-mailman source examines the envelope recipient but not the To header field, which is often bogus in spam email.

The following group of sources may be used only in incoming filter files.

body [ -case ] <regular_expression>
headers [ -case ] <regular_expression>
The body and headers sources expect a match field that is a regular expression as defined in Python's re module. The body source matches against the body of the message while the headers matches against the header fields.

Because regular expressions may include spaces, you must surround the regular expressions with quotation marks. You may use either single quotes (') or double quotes (") as long as you use the the same one at both the beginning and the end.

If you need to match a quote in your regular expression, simply use the other style of quotes to surround the expression or escape the embedded quote with a backslash (\).

The regular expression match is case-insensitive by default. If you want a case-sensitive match, specify the -case flag.

body-file [ -case ] [-optional ] <regexp_file>
headers-file [ -case ] [-optional ] <regexp_file>
The body-file and headers-file sources expect the match field to contain the filename of a textfile containing one or more regular expressions as defined in Python's re module. The body-file source matches against the body of the message while the headers-file matches against the header fields. The format of the regular expression file is documented below.

The regular expression match is case-insensitive by default. If you want a case-sensitive match, specify the -case flag.

If the -optional flag is given, the non-existence of the file is not an error. If the file should exist, don't specify this flag; the parser will log an error and will defer the mail so that you have a chance to fix the problem.

size < <size_in_bytes | >size_in_bytes >
The size source expects a comparison operator and a number of bytes to compare to the size of the message. Only the `<' and `>' operators are supported. There must not be any whitespace between the operator and the number.

Miscellaneous Notes

Email Addresses
In addition to explicit email addresses, you can use expressions based on UNIX shell-style wildcard characters anywhere an email address is expected.

NOTE: Wildcard characters are not recognized in a CDB or DBM file.

The special characters are:

Characters(s)    Description
-------------    -----------
*                Matches everything.
?                Matches any single character.
[seq]            Matches any character in seq.
[!seq]           Matches any character not in seq.

In addition, `@=' (a custom rule) will expand to match both @ and @*.

Here are some common examples:
# match only jdoe@domain.dom jdoe@domain.dom # match anyone@domain.dom, but not anyone@sub.domain.dom *@domain.dom # match anyone@sub.domain.dom, but not anyone@domain.dom *@*.domain.dom # match both anyone@domain.dom, and anyone@sub.domain.dom *@=domain.dom
NOTE: To match the empty envelope sender such as bounce messages are sent with, use <> as the expression.

Email Address Files
Email address files are textfiles containing an email address on each line. When using the from-file and to-file sources, the textfile is searched sequentially, with the first match terminating the search. Wildcarded email addresses are supported in textfiles.

Address files may contain an optional second field on each line that specifies an action (ok, drop, bounce, etc.). If the action is specified, it overrides the action given in the filter rule.

Auto- Database Flags
If you have lengthy email address textfiles, you might want to consider using the much faster hashed databases instead. The address files used by the auto-building hashed database feature are the same email address textfiles documented above with the sole exception that wildcards are not supported.

The -autocdb and -autodbm arguments are intended to ease the use of CDB/DBM lists in TMDA by automatically rebuilding the CDB or DBM file as necessary. This gives you the performance advantages of hashed databases without the hassle of having to manually maintain them. With the -auto* arguments, TMDA will rebuild the database if it doesn't exist or if its timestamp is older than its source file. If the rebuild fails for some reason, TMDA will fall back to matching from the textfile instead.

Before you try the CDB version of this feature, make sure you have the python-cdb extension module installed.

Database Files
CDB and DBM files are hashed databases. TMDA can look up email addresses in these files. Lookup in these files is much faster than in a textfile. On the other hand, wildcards are not supported in database files -- only in textfiles.

In a CDB or DBM, the keys should be the email addresses to match, and their corresponding values (or records) should be empty unless you want to override the action specified in the filter file.

CDB or DBM files can be created outside of TMDA and merely referenced by your filter files (use the *-cdb and *-dbm filter rules) or can be automatically created by TMDA if you use the -autocdb or -autodbm flags and the *-file rules.

If you wish to explore CDB databases, make sure you have the python-cdb extension module installed.

Regular Expression Files
A regular expression textfile is simply a text file with a regular expression on each line. The file is read sequentially and each regular expression is used to attempt a match. As soon as there is a match, the search stops.

Because regular expressions may include spaces, you must surround the regular expressions with quotation marks. You may use either single quotes (') or double quotes (") as long as you use the the same one at both the beginning and the end.

If you need to match a quote in your regular expression, simply use the other style of quotes to surround the expression or escape the embedded quote with a backslash (\).