The project page is here.
Download fdm 1.5 with this link.
A mailing list is available, see this page.
fdm
Introduction
fdm is a program to fetch mail and deliver it in various ways depending on a user-supplied ruleset. Mail may be fetched from stdin, IMAP or POP3 servers, or from local maildirs, and filtered based on whether it matches a regexp, its size or age, or the output of a shell command. It can be rewritten by an external process, dropped, left on the server or delivered into maildirs, mboxes, to a file or pipe, or any combination. fdm is designed to be lightweight but powerful, with a compact but clear configuration syntax. It is primarily designed for single-user uses but may also be configured to deliver mail in a multi-user setup. In this case, it uses privilege separation to minimise the amount of code running as the root user.Table of contents
1 Installation 2 Quick start 3 The configuration file 3.1 Including other files 3.2 Macros 3.3 Testing macros 3.4 Shell commands 4 Invoking fdm 4.1 Temporary files 4.2 Command line arguments 4.3 Running from cron 4.4 The lock file 4.5 Testing and debugging 5 Fetching mail 5.1 Mail tags 5.2 POP3 and POP3S 5.3 SSL certificate verification 5.4 The .netrc file 5.5 IMAP and IMAPS 5.6 IMAP or POP3 over a pipe or ssh 5.7 stdin and local mail 5.8 From maildirs and mboxes 5.9 Using NNTP and NNTPS 5.10 New or old mail only 6 Defining actions 6.1 Drop and keep 6.2 Maildirs 6.3 Mboxes 6.4 SMTP 6.5 Write, pipe, exec and append 6.6 stdout 6.7 Rewriting mail 6.8 Adding or removing headers 6.9 Tagging 6.10 Compound actions 6.11 Chained actions 7 Filtering mail 7.1 Nesting rules 7.2 Lambda actions 7.3 The all condition 7.4 Matched and unmatched 7.5 Matching a regexp 7.6 Matching bracket expressions 7.7 Matching by age or size 7.8 Using a shell command 7.9 Attachments 7.10 Matching tags 7.11 Using caches 7.12 Cache commands 8 Setting options 9 Archiving and searching mail 10 Using fdm behind a proxy 11 Bug reports and queries 12 Frequently asked questions1 Installation
fdm depends on the Trivial Database library (TDB), available at: http://sourceforge.net/projects/tdb/ Ensure it is installed, then download the source tarball and build fdm with: $ tar -zxvf fdm-?.?.tar.gz $ cd fdm-?.? $ make Then run 'make install' to install fdm to the default locations under /usr/local. The PREFIX environment variable may be set to specify an alternative installation location: $ export PREFIX=/opt # defaults to /usr/local $ sudo make install If being run as root, fdm requires a user named "_fdm" to exist. It will drop privileges to this user and its primary group. The user may be added on OpenBSD with, for example: # useradd -u 999 -s /bin/nologin -d /var/empty -g=uid _fdm It is not necessary to add a user if fdm is always started by a non-root user. fdm can be built to use PCRE rather than standard regexps. To do so, add -DPCRE to the make command: $ make -DPCRE Or PCRE=1 if using GNU make: $ make PCRE=12 Quick start
A simple ~/.fdm.conf file for a single user fetching from POP3, POP3S and IMAP accounts and delivering to one maildir may look similar to: # Set the maximum size of mail. set maximum-size 128M # An action to save to the maildir ~/mail/inbox. action "inbox" maildir "%h/mail/inbox" # Accounts: POP3, POP3S and IMAP. Note the double escaping of the '\' # character in the password. If the port is omitted, the default # ("pop3", "pop3s", "imap" or "imaps" in the services(5) db) is used. account "pop3" pop3 server "my.pop3.server" user "my-username" pass "my-password-with-a-\\-in-it" account "pop3s" pop3s server "pop.googlemail.com" port 995 user "my-account@gmail.com" pass "my-password" # If the 'folder "my-folder"' argument is omitted, fdm will fetch mail # from the inbox. account "imap" imap server "my.imap.server" user "my-username" pass "my-password" folder "my-folder" # Discard mail from Bob Idiot. Note that the regexp is an extended # regexp, and case-insensitive by default. This action is a "lambda" or # unnamed action, it is defined inline as part of the match rule. match "From:.*bob@idiot\\.net" action drop # Match all other mail and deliver using the 'inbox' action. match all action "inbox" A simple initial configuration file without filtering, perhaps to replace fetchmail or getmail delivering to maildrop, may look similar to: # Set the maximum size of mail. set maximum-size 128M # Action to pipe directly to maildrop. action "maildrop" pipe "/usr/local/bin/maildrop" # Account definitions. account .... # Send all mail to maildrop. match all action "maildrop" To run fdm every half hour from cron, add something like this: */30 * * * * /usr/local/bin/fdm -l fetch See the fdm.conf(5) man page or the rest of this manual for more detail of the configuration file format.3 The configuration file
fdm is controlled by its configuration file. It first searches for a .fdm.conf file in the invoking user's home directory. If that fails, fdm attempts to use /etc/fdm.conf. The configuration file may also be specified using the '-f' command line option, see the section on that subject below. This section gives an overview of the configuration file syntax. Further details of syntax, and specific keywords, are covered in later sections. The configuration file has the following general rules: - Keywords are specified as unadorned lowercase words: match, action, all. - Strings are enclosed in double quotes (") or single quotes ('). In double quoted strings, double quotes may be included by escaping them using the backslash character (\). Backslashes must also be escaped ("\\") - this applies to all such strings, including regexps and passwords. The special sequence '\t' is replaced by a tab character. In single quoted strings no escaping is necessary, but it is not possible to include a literal ' or a tab character. - Comments are prefixed by the hash character (#) and continue to the end of the line. - Whitespace is largely ignored. Lines may generally be split, concatenated or indented as preferred. - Lists are usually specified as 'singular item' or 'plural { item item }', for example: 'user "nicholas"', 'users { "nicholas" "bob" }'. The singular/plural distinction is not required, it is recommended only to aid readability: 'user { "nicholas "bob" }' is also accepted. - Regexps are specified as normal strings without additional adornment other than the "s (not wrapped in /s). All regexps are extended regexps. They are case insensitive by default but may be prefixed with the 'case' keyword to indicate case sensitivity is required. - Strings may be concatenated using plus: "a" + "b" is the same as "ab". This is most useful to wrap strings across multiple lines. Definition/option lines generally follow the following basic form: <keyword> <name or command> <parameters> Example lines that may appear in a configuration file are: # This is a comment. set lock-types flock account "stdin" disabled stdin action "strip-full-disclosure" rewrite "sed 's/^\\(Subject:.*\\)\\[Full-disclosure\\] /\\1/'" match "^X-Mailing-List:.*linux-kernel@vger.kernel.org" in headers or "^(To:|Cc:):.*@vger.kernel.org" in headers action "linux-kernel"3.1 Including other files
The fdm configuration may be split into several files. Additional files may be referenced using the 'include' keyword: include "my-include-file.conf" include "/etc/fdm.d/shared-conf-1.conf"3.2 Macros
Macros may be defined and used in the configuration file. fdm makes a distinction between macros which may hold a number (numeric macros) and those that hold a string (string macros). Numeric macros are prefixed with the percentage sign (%) and string by the dollar sign ($). Macros are defined using the equals operator (=): %nummacro = 123 $strmacro = "a string" Macros may then be referenced in either a standalone fashion anywhere a string or number is expected, depending on the type of macro: $myfile = "a-file" include $myfile %theage = 12 match age < %theage action "old-mail" Or embedded in a string by enclosing the macro name in {}s: $myfile2 = "a-file2" include "/etc/${myfile2}" %anum = 57 include "/etc/file-number-%{anum}" Macros are not substituted in strings specified using single-quotes.3.3 Testing macros
The 'ifdef', 'ifndef' and 'endif' keywords may be used to include or omit sections of the configuration file depending on whether a macro is defined. An 'ifdef' is followed by a macro name (including $ or % type specifier) and if that macro exists, all following statements up until the next endif are evaluated (accounts created, rules added, and so on), otherwise they are skipped. 'ifndef' is the inverse: if the macro exists, the statements are skipped, otherwise they are included. An example is: ifdef $dropeverything match all action drop endif These keywords are particularly useful in conjunction with the '-D' command line option. Any statements between 'ifdef'/'ifndef' and 'endif' must still be valid syntax.3.4 Shell commands
The value of a shell command may be used at any point in the configuration file where fdm expects a string or number. Shell commands are invoked by enclosing them in $() or %(). They are executed when the configuration file is parsed and if $() is used, any output to stdout is treated as a literal string (as if the output was inserted directly in the file enclosed in double quotes); %() attempts to convert the output to a number. For example: $mytz = $(date +%Z) %two = %(expr 1 + 1) $astring = "abc" + $(echo def) Parts of the command within double quotes (") are subject to tag and macro replacement as normal (so it is necessary to use %% if a literal % is required, see the section on tags below); parts outside double quotes or inside single quotes are not.4 Invoking fdm
fdm may be invoked manually from the command line or regularly using a program such as cron(8).4.1 Temporary files
As each mail is being processed, it is stored in a temporary file in /tmp, or if the TMPDIR environment variable exists in the directory it points to. fdm tries to queue a number of mails simultaneously, so that older can be delivered while waiting for the server to provide the next. The maximum length of the queue for each account is set by the 'queue-high' option (the default is two) and the maximum mail size accepted by the 'maximum-size' option (the default is 32 MB). In addition, the 'rewrite' action requires an additional temporary mail. Although fdm will fail rather than dropping mail if the disk becomes full, users should bear in mind the possibility and set the size of the temporary directory and the fdm options according to their needs.4.2 Command line arguments
The fdm command has the following synopsis: fdm [-klmnqv] [-f conffile] [-u user] [-a account] [-x account] [-D name=value] [fetch | poll | cache ...] The meaning of the flags are covered in the fdm(1) man page, but a brief description is given below. The flags are also mentioned at relevant points in the rest of this document. Flag Meaning -k Keep all mail (do not delete it from the server). This is useful for testing delivery rules without risking mail ending up permanently in the wrong place. -l Log to syslog(3) using the 'mail' facility rather than outputting to stderr. -m Ignore the lock file. -n Run a syntax check on the configuration file and exit without fetching any mail. -q Quiet mode. Don't print anything except errors. -v Print verbose debugging output. This option may be specified multiple times for increasing levels of verbosity. Useful levels are -vv to display the result of parsing the configuration file, and -vvvv to copy all traffic to and from POP3 or IMAP servers to stdout (note that -l disables this behaviour). -f conffile Specify the path of the configuration file. -u user Use 'user' as the default user for delivering mail when started as root. -a account Process only accounts with a name matching the given pattern. Note that fnmatch(3) wildcards may be used to match multiple accounts with one option, and that the option may be specified multiple times. -x account Process all accounts except those that match the given pattern. Again, fnmatch(3) wildcards may be used, and the -x option may be specified multiple times. -D name=value Define a macro. The macro name must be prefixed with '$' or '%' to indicate if it is a string or numeric macro. Macros defined on the command line override any macros with the same name defined in the configuration file. If -n is not specified, the flags must be followed by one of the keywords 'fetch' or 'poll' or 'cache'. The 'fetch' keyword will fetch and deliver mail, the 'poll' keyword print an indication of how many mails are present in each account, and the 'cache' keyword is followed by one of a set of cache commands used to manipulate caches from the command-line (see the sections on caches below). 'fetch' or 'poll' or 'cache' may be abbreviated. Examples: $ fdm -v poll $ fdm -vvnf /etc/my-fdm.conf $ fdm -lm -a pop3\* fetch $ fdm -x stdinacct fetch # fdm -u nicholas -vv f4.3 Running from cron
To fetch mail regularly, fdm must be run from cron. This line in a crontab(5) will run fdm every 30 minutes: */30 * * * * /usr/local/bin/fdm -l fetch The '-l' option sends fdm's output to syslog(3) rather than having cron mail it. To keep a closer eye, adding '-v' options and removing '-l' will have debugging output mailed by cron, or, using a line such as: */30 * * * * fdm -vvvv fetch >>/home/user/.fdm.log 2>&1 Will append extremely verbose fdm output to the ~/.fdm.log file. Note that this log file can become pretty large, so another cronjob may be required to remove it occasionally!4.4 The lock file
fdm makes use of a lock file to prevent two instances running simultaneously. By default, this lock file is .fdm.lock in the home directory of the user who runs fdm, or /var/db/fdm.lock for root. This default may be overridden in the configuration file with the 'set lock-file' command: set lock-file "/path/to/my/lock-file" Or disabled altogether by being set to the empty string: set lock-file "" The '-m' command line option may be used to force fdm to ignore the lock file and run regardless of its existence and without attempting to create it.4.5 Testing and debugging
fdm has some features to assist with testing and debugging a ruleset: The '-n' command line option. This is particularly useful in conjunction with '-vv', for example: $ cat test.conf account "pop3" pop3 server "s" user "u" pass "p" action "rw" rewrite "sed 's/\\(Subject:.*\\)\\[XYZ\\]/\1/'" action "mbox" mbox "%h/INBOX" match all actions { "rw" "mbox" } $ fdm -vvnf test.conf version is: fdm 0.6 (20061204-1433) starting at: Tue Dec 5 15:45:41 2006 user is: nicholas, home is: /home2/nicholas loading configuration from test.conf added account: name=pop3 fetch=pop3 server "s" port pop3 user "u" added action: name=rw deliver=rewrite "sed 's/\(Subject:.*\)\[XYZ\]/1/'" added action: name=mbox deliver=mbox "%h/INBOX" finished file test.conf added rule: actions="rw" "mbox" matches=all configuration loaded locking using: flock headers are: "to" "cc" domains are: "yelena" using tmp directory: /tmp Looking at the output, the parsed strings used by fdm can be seen, and it is possible to spot that an escape character has been missed in the command. If '-vvvv' is used, fdm will print all data sent to and received from remote servers to stdout. Note that this is disabled if the '-l' option is given, and includes passwords, usernames and hostnames unmodified. The 'fdm-sanitize' script provided with fdm may be used to remove passwords and usernames from this output, either while it is being collected: fdm -vvvv -a testacct f 2>&1|./fdm-sanitize|tee my-output Or afterwards: ./fdm-sanitize <vvvv-output >my-output Since fdm fetches multiple accounts simultaneously, which may intersperse debugging output, it is recommended to fetch each account seperately if running the output through fdm-sanitize. If this is not done, it may not be able to detect all usernames or passwords. The '-k' command line option (and the 'keep' keywords on actions and accounts, covered later) prevent fdm from deleting any mail after delivery. This may be used to perform any number of test deliveries without risk of losing mail.5 Fetching mail
fdm fetches mail from a set of 'accounts', defined using the 'account' keyword. Each account has a name, a type, a number of account specific parameters and a couple of optional flags. The general form is: account <name> [<users>] [disabled] <type> [<parameters>] [keep] The <name> item is a string by which the account is referred in filtering rules, log output and for the '-a' and '-x' command line options. The <users> portion specifies the default users to use when delivering mail fetched from this account as root. It has the same syntax as discussed in detail in the section below on defining actions. If the optional 'disabled' keyword is present, fdm ignores the account unless it is specified on the command line using the '-a' flag. The optional 'keep' keyword instructs fdm to keep all mail from this account (not delete it from the server) regardless of the result of the filtering rules. The <type> item may be one of: 'pop3', 'pop3s', 'imap', 'imaps', 'stdin', 'maildir' or 'maildirs'.5.1 Mail tags
As mail is processed by fdm, it is tagged with a number of name/value pairs. Some tags are added automatically, and mail may also be tagged explicitly by the user (see the later tagging section). Tags may be inserted in strings in a similar manner to macros, except tags are processed when the string is used rather than always as the configuration file is parsed. A tag's value is inserted by wrapping its name in %[], for example: match string "%[account]" to "myacct" action "myacctact" Most of the default tags have a single-letter shorthand which removes the needs for the []s: match string "%a" to "myacct" action "myacctact" Including a nonexistent tag in a string is equivalent to including a tag with an empty value, so "abc%[nonexistent]def" will be translated to "abcdef". The automatically added tags are: Name Shorthand Replaced with account %a The name of the account from which the mail was fetched. home %h The delivery user's home directory. uid %n The delivery user's uid. action %t The name of the action the mail has matched. user %u The delivery user's username. hour %H The current hour (00-23). minute %M The current minute (00-59). second %S The current second (00-59). day %d The current day of the month (00-31). month %m The current month (01-12). year %y The current year as four digits. year2 The current year as two digits. dayofweek %W The current day of the week (0-6, Sunday is 0). dayofyear %Y The current day of the year (000-365). quarter %Q The current quarter (1-4). rfc822date The current time in RFC822 date format. mail_hour The hour from the mail's date header, converted to local time, if it exists and is valid, otherwise the current time. mail_minute The minute from the mail's date header. mail_second The second from the mail's date header. mail_day The day from the mail's date header. mail_month The month from the mail's date header. mail_year The year from the mail's date header as four digits. mail_year2 The same as two digits. mail_rfc822date The mail date in RFC822 format. hostname The local hostname. In addition, the shorthand %% is replaced with a literal %, and %1 to %9 are replaced with the result of any bracket expressions in the last regexp (see later section on regexps). A leading ~ or ~user is expanded in strings where a path or command is expected. Some accounts add additional tags, discussed below. Tags are replaced in almost all strings (including those in single-quotes!), some when the configuration file is parsed and some when the string is used.5.2 POP3 and POP3S
Mail may be fetched from a POP3 account. A POP3 account is defined by specifying the following parameters: the server host and optionally port, and optionally the user name and password. If the port is not specified, the default port ('pop3' in the services(5) database) is used. If the user name, password, or both is omitted, fdm attempts to look it up the .netrc file, see the next section for details. Examples of a POP3 account definition are: account "pop3acct" pop3 server "pop.isp.com" user "bob" pass "pass" account "gmx" pop3 server "pop.gmx.net" port 110 user "jim" pass "pass" account "acct" pop3 server "10.0.0.1" port "pop3" user "nicholas" keep account "lycos" disabled pop3 server $localserver port 10110 pass "password" Note that the server string is enclosed in double quotes even if it is an IP, and don't forget to escape any " and \ characters in passwords! fdm will attempt to use APOP to obscure the password, if the server offers it. If the server advertises itself as supporting APOP but subsequently refuses to accept it, fdm will not retry with a cleartext password. Use of APOP can be disabled for an account using the 'no-apop' flag, for example: account "acct" pop3 server "server" user "bob" pass "pass" no-apop POP3S is specified in exactly the same way, except using the 'pop3s' keyword for the type, and the default port is 'pop3s' rather than 'pop3': account "pop3sacct" pop3s server "pop.isp.com" user "bob" pass "pass" POP3 accounts automatically tag mail with 'server' and 'port' tags, with the value of the server and port attributes exactly as specified in the account definition. A 'server_uid' tag is also added with the server unique id (UIDL). POP3 adds 'lines', 'body_lines' and 'header_lines' tags with the number of lines in the complete mail and its body and header. These tags are not updated to reflect any changes made to the mail by fdm rules.5.3 SSL certificate verification
fdm can verify SSL certificates before collecting mail from an SSL server. This is enabled globally with the 'verify-certificates' option: set verify-certificates And may be disabled per-account using the 'no-verify' keyword (this applies to both POP3S and IMAPS accounts): account "pop3sacct" pop3s server "pop.isp.com" no-verify For an introduction to SSL, see: http://httpd.apache.org/docs/2.0/ssl/ssl_intro.html A cert bundle is required to verify SSL certificate chains. For more information see: http://lynx.isc.org/current/README.sslcerts A pregenerated bundle is available courtesy of the MirOS project: http://cvs.mirbsd.de/src/etc/ssl.certs.shar5.4 The .netrc file
If the user name or password is omitted in POP3 or IMAP account definitions, fdm will attempt to look it up in the .netrc file in the invoking user's home directory. The .netrc file format is shared with ftp(1) and some other programs. It consists of a number of 'machine' sections and optionally one 'default' section containing a username ('login') and password for that host. fdm accepts entries only if the machine name matches the POP3 or IMAP server string exactly. If no matches are found and a 'default' section exists, it is used. An example .netrc file is: machine "my.mail-server.com" login "nicholas" password "abcdef" machine "pop.googlemail.com" password "pass1" default login "bob" password "moo" fdm will abort if the .netrc file is world-writable or world-readable.5.5 IMAP and IMAPS
IMAP and IMAPS accounts are defined using exactly the same syntax as for POP3 and POP3S, aside from using the 'imap' or 'imaps' keywords and that the default port is 'imap' or 'imaps'. There is also an additional, optional 'folder' option to specify the folder from which mail should be fetched. If omitted, the folder defaults to the inbox. Note that with IMAP and IMAPS, mail is still removed from the server unless the 'keep' option is given, or the '-k' command line option used. Examples of IMAP and IMAPS accounts include: account "imapacct" imap server "imap.server.ca" user "bob" pass "pass" account "oldimap" disabled imaps server "192.168.0.1" port 10993 user "nicholas" pass "pass" folder "Saved" account "workspam" disabled imap server "my-work.ath.cx" user "Nicholas" folder "Junk" As with POP3, IMAP adds the 'server', 'port', 'server_uid' and the three line count tags to mail.5.6 IMAP or POP3 over a pipe or ssh
Mail may be fetched using IMAP or POP3 via a pipe. This is particularly useful for fetching mail over ssh using public keys. For IMAP, a user and password may be supplied, but fdm will only use them if the server asks. If the connection is preauthenticated, the user and password are unnecessary. For POP3, a user and password must be supplied as usual: due to the lack of server name, it cannot be read from the .netrc file. Communication takes place via the pipe program's stdin and stdout. If any output is found on stderr, fdm will print it (or log it with '-l'). Examples are: account "imapssh" imap pipe "ssh jim@myhost /usr/local/libexec/imapd" account "imapssh2" imap pipe "/usr/bin/whatever" user "bob" pass "bah" account "pop3local" pop3 pipe "/usr/local/bin/ipop3d" user "me" pass "foo"5.7 stdin and local mail
fdm may be configured to fetch mail from stdin, by specifying an account of type 'stdin', for example: account "stdin" disabled stdin This is most useful to have fdm behave as a mail delivery agent. To configure it for single-user use with sendmail, the simplest method it to add: "|/usr/local/bin/fdm -m -a stdin fetch" To the user's ~/.forward file (including the double quotes). Note the use of '-m' to prevent stdin delivery from interfering with any normal cronjob, and '-a' to specify that only the disabled "stdin" account should be fetched. stdin accounts add the three line count tags described in the POP3 section.5.8 From maildirs and mboxes
Fetching from maildirs allows fdm to be used to filter mail on the local machine. This is covered more detail in the later section on archiving and searching. Maildir accounts are specified as follows: account "mymaildir" maildir "/path/to/dir" account "mymaildirs" maildirs { "/path/to/dir1" "/path/to/dir2" } Shell glob wildcards may be included in the path names to match multiple maildirs, but every directory found must be a valid maildir. Maildir accounts tag mail with a 'maildir' tag which is the basename of the maildir. Fetching from mboxes is similar: account "mybox" mbox "/path/to/mbox" account "mymboxes" mboxes { "/path/to/mbox1" "/path/to/mbox2" } Note that if an mbox is modified (mail is dropped from it), sufficient disk space is required to create a temporary copy of the entire mbox.5.9 Using NNTP and NNTPS
fdm can fetch news messages from a news server using NNTP or NNTPS. News accounts are specified like so: account "news1" nntp server "news.server.sk" port 119 group "comp.unix.bsd.openbsd.misc" cache "%h/.fdm.cache/%[group]" account "mynews" nntps server "ssl.news.server" port "nntps" groups { "alt.test" "alt.humor.best-of-usenet" } cache "%h/.fdm.cache" The cache is a file used to store details of the last article fetched. If only one group is supplied in the account definition, %[group] tags are replaced by the name of the group in the cache path. If multiple groups are provided, %[group] is removed. Note that whether a message is kept or deleted is irrelevent to NNTP, articles are always left on the server. The index and message-id of the last article is recorded in the cache file so that older articles are skipped when the a newsgroup is again fetched. This happens regardless of any 'keep' keywords or the '-k' command line option. As with POP3 and IMAP, NNTP accounts add the 'server' and 'port' tags to mail. In addition, a 'group' tag is added with the group name. This can ensure articles are matched purely on the group they are fetched from (trying to do this using headers is unreliable with cross-posted articles). For example: match account "news" { match string "%[group]" to "comp.lang.c" action "news-%[group]" match string "%[group]" to "comp.std.c" action "news-%[group]" match all action drop }5.10 New or old mail only
With POP3 and IMAP, fdm can be set up to fetch only new or old mail. For POP3 this is achieved by recording the current state of the server in a cache file, which is updated as each mail is fetched. For IMAP it makes use of the 'seen' server flag which is updated by the server after each mail is fetched. These options are specified as in the following examples. For POP3: account "name" pop3 server "blah" new-only cache "~/.fdm-pop3-cache" account "acct" pop3s server "my-server" user "bob" new-only cache "my-server-pop3-cache" no-apop And for IMAP: account "imap" imap server "blah" new-only account "sslimap" imaps server "imaps.somewhere" user "user" pass "pass" old-only no-verify Note that currently, when using this with IMAP, the server is permitted to flag the mail as 'seen' before fdm has successfully delivered it, so there is no guarantee that mail so marked has been delivered, only that it has been fetched.6 Defining actions
An action is a particular command to execute on a mail when it matches a filtering rule (see the next section on filtering mail). Actions are named, similar to accounts, and have a similar form: action <name> [<users>] <type> <parameters> The <users> item may be either: - the keyword 'user' followed by a single username string or uid, such as: user "nicholas" user 1000 - the keyword 'users' followed by a list of users in {}s, for example: users { 1001 "nicholas" } - the keyword 'from-headers', that will attempt to ascertain a set of users from the mail headers (see the 'domains' and 'headers' options in the section on setting options). If users are specified, the action will be run once for each user, with fdm changing to that user before executing the action. Note that fdm will execute the action once for each user even when not started as root, but will not be able to change to the user. The user keyword is primarily of use in multiuser configurations. If users are present on an action, they override any specified by the account defintion. If running as root and no user is specified on either the action or on the filtering rule (see the section on filtering below), the default user is used, see the '-u' command line option and the 'default-user' option in the setting options section6.1 Drop and keep
The simplest actions are the 'drop' and 'keep' actions. They have no parameters and are specified like this: action "mydropaction" drop action "mykeepaction" keep The 'drop' action arranges for mail to be dropped when rule evaluation is complete. Note that using 'drop' does not stop further evaluation if the filtering rule contains a 'continue' keyword, and it may be overridden by a 'keep' option on the account or by the '-k' flag on the command line. The 'keep' action is similar to 'drop', but it arranges for the mail to be kept once rule evaluation is complete, rather than dropped.6.2 Maildirs
Mails may be saved to a maildir through a 'maildir' action, defined like so: action "mymaildiraction" maildir "/path/to/maildir" If any component of the maildir path does not exist, it is created, unless the no-create option is specified. Mails saved to a maildir are tagged with a 'mail_file' tag containing the full path to the file in which they were saved.6.3 Mboxes
An action to deliver to an mbox is defined in the same way as for a maildir: action "mymboxaction" mbox "/path/to/mbox" The same % tokens are replaced in the path. If the mbox does not exist, it is created. Mboxes may optionally be gzip compressed by adding the 'compress' keyword: action "mymboxaction" mbox "/path/to/mbox" compress fdm will append .gz to the mbox path (if it is not already present) and append compressed data. If the mbox exists but is not already compressed, uncompressed data will be appended. As with maildirs, if any component of the mbox path does not exist, it is created, unless the no-create option is set. Mails saved to an mbox are tagged with an 'mbox_file' tag with the path of the mbox.6.4 SMTP
An action may be defined to pass mail on over SMTP. The server host must be specified and optionally the port and string to pass to the server with the RCPT TO and MAIL FROM commands. If the port is not specified it defaults to "smtp". Examples include: action "mysmtpaction" smtp server "smtp.server" action "mysmtpaction" smtp server "smtp.server" port 587 action "mysmtpaction" smtp server "smtp.server" port "submission" from "bob@server.com" action "mysmtpaction" smtp server "smtp.server" to "me@somewhere"6.5 Write, pipe, exec and append
Actions may be defined to write or append a mail to a file, to pipe it to a shell command, or merely to execute a shell command. The append action appends to and write overwrites the file. % tokens are replaced in the file or command as for maildir and mbox actions. Examples are: action "mywriteaction" write "/tmp/file" action "myappendaction" append "/tmp/file" action "mypipeaction" pipe "cat > /dev/null" action "domaildirexec" exec "~/.fdm.d/my-special-script %[mail_file]" Pipe and exec commands are run as the command user (by default the user who invoked fdm).6.6 stdout
fdm can write mails directly to stdout, using the 'stdout' action: action "so" stdout6.7 Rewriting mail
Mail may be altered by passing it to a rewrite action. This is similar to the pipe action, but the output of the shell command to stdout is reread by fdm and saved as a new mail. This is useful for such things as passing mail through a spam filter or removing or altering headers with sed. Note that rewrite only makes sense on filtering rules where the continue keyword is specified, or where multiple actions are used (see the next section for details of this). Possible rewrite action definitions are: action "myspamaction" rewrite "bmf -p" action "mysedaction" rewrite "sed 's/x/y/'"6.8 Adding or removing headers
Simple actions are provided to add a header to a mail: action "lines" add-header "Lines" value "%[lines]" Or to remove all instances of a header from mail: action "del-ua" remove-header "user-agent" action "rmhdr" remove-header "x-stupid-header" action "remove-headers" remove-headers { "X-*" "Another-Header" }6.9 Tagging
Mails may be assigned one of more tags manually using the tag action type. For example, match account "my*" tag "myaccts" match "^User-Agent:[ \t]*(.*)" tag "user-agent" value "%1" The tag is attached to the mail with the specified value, or no value if none is provided.6.10 Compound actions
Compound actions may be defined which perform multiple single actions. They are similar to standard single actions but multiple actions are provided using {}. For example, action "multiple" { add-header "X-My-Header" value "Yay!" mbox "mbox2" } action "myaction" users { "bob" "jim" } { rewrite "rev" maildir "%h/%u's maildir" } Compound action are executed from top-to-bottom, once for each user. Note that the effects are cumulative: the second example above would deliver a mail rewritten once to 'bob' and rewritten again (ie, twice) to 'jim'. If this is not desired, seperate actions must be used.6.11 Chained actions
An action may call other named actions by reusing the 'action' keyword: action "abc" action "def" action "an_action" { rewrite "rev" action "another_action" action "yet_more_actions" } There is a hard limit of five chained actions in a sequence to prevent infinite loops.7 Filtering mail
Mail is filtered by defining a set of filtering rules. These rules tie together mail fetched from an account and passed to one or more actions. Rules are evaluated from top-to-bottom of the file, and evaluation stops at the first matching rule (unless the continue keyword is specified). The general form of a filtering rule is: match <conditions> [<users>] <actions> [continue] The optional <users> item is specified as for an action definition. If users are specified on a filtering (match) rule, they override any specified on the action or account. The <conditions> item is set of conditions against which the match may be specified, each condition returns true or false. Conditions are described in the next few sections. Aside from the 'all' condition, which is a special case, conditions may be chained as an expression using 'and' and 'or', in which case they are evaluated from left to right at the same precedence, or prepended with 'not' to invert their outcome. The <actions> item is a list of actions to execute when this rule matches. It is in the same list format: 'action "name"' or 'actions { "name1" "name2" }'. It may also be a lambda (inline) action, see the section below. If a rule with the 'continue' keyword matches, evaluation does not stop after the actions are executed, instead subsequent rules are matched.7.1 Nesting rules
Filtering rules may be nested by using the special form: match <conditions> [<accounts>] { match ... } If the conditions on the outer rule match, the inner rules are evaluated. If none of the inner rules match (or they all specify the 'continue' keyword) evaluation continues outside to rules following the nested rule, otherwise it stops.7.2 Lambda actions
Lambda actions are unnamed actions included inline as part of the filtering rule. This can be convenient for actions which do not need to be used multiple times. Lambda actions are specified as a combination of the rule and an action definition. For example: match all action maildir "mymaildir" match all actions { rewrite "rev" tag "reversed" } continue7.3 The all condition
The all condition matches all mail. Examples include: match all action "default" match all rewrite "rewaction" continue7.4 Matched and unmatched
The matched and unmatched conditions are used to match mail that has matched or has not matched previous rules and been passed on with the 'continue' keyword. For example, match "myregexp" action "act1" continue # This rule will match only mails that also matched the first. match matched action "act2" # This rule will match only mails that matched neither of the first two. match unmatched action "act3"7.5 Matching by account
The account condition matches a list of accounts from which the mail was fetched. It is specified as either a single account ('account "name"') or a list of accounts ('accounts { "name1" "name2" }'). fnmatch(3) wildcards may also be used. Examples include: match "blah" accounts { "pop3" "imap" } action "go!" match matched and account "myacc" action drop7.6 Matching a regexp
Matching against a regexp is the most common form of condition. It takes the following syntax: [case] <regexp> [in headers|in body] The 'case' keyword instructs fdm to match the regexp case sensitively rather than the default of case insensitivity. The 'in headers' or 'in body' keywords make fdm search only the headers or body of each mail, the default is to match the regexp against the entire mail. Any multiline headers are unwrapped onto a single line before matching takes place and the process reversed afterwards. The regexp itself is an extended regexp specified as a simple string, but care must be taken to escape \s and "s properly. Examples include: match "^From:.*bob@bobland\\.bob" in headers account "pop3" action "act" match ".*YAHOO.*BOOTER.*" in body action "junk"7.7 Matching bracket expressions
The results of any bracket expressions within the last regexp match are remembered, and may be made use of using the 'string' condition, or used to construct an action name, maildir or mbox path, etc. The bracket expressions may be substituted using the %0 to %9 tokens. For example, match "^From:.*[ \t]([a-z]*)@domain" in headers action "all" continue match string "%1" to "bob.*" action "bobmail" match "^From:.*[ \t]([a-z]*)@domain" in headers action "all" continue match all action "%1mail" This is particularly useful in combination with nested rules (see later): bracket expressions in a regexp on the outer rule may be compared on inner rules. Note that %0 to %9 are used only for 'regexp' rules. Regexps that are part of 'command' rules use the 'command0' to 'command9' tags.7.8 Matching by age or size
Mail may be matched based on its age or size. An age condition is specified as follows: age [<|>] <age> [hours|minutes|seconds|days|months|years] If '<' is used, mail is matched if it is younger than the specified age. If '>', if it is older. The <age> item may be a simple number of seconds, or suffixed with a unit. Examples are: match age < 3 months actions { "act1" "act2" } match age > 100 hours action "tooold" The age is extracted from the 'Date' header, if possible. To match mails for which the header was invalid, the following form may be used: match age invalid action "baddate" The size condition is similar: size [<|>] <size> [K|KB|kilobytes...] Where <size> is a simple number in bytes, or suffixed with 'K', 'M' or 'G' to specify a size in kilobytes, megabytes or gigabytes, such as: match size < 1K action "small" match size > 2G action "whoa"7.9 Using a shell command
Mail may be matched using the result of a shell command. This condition follows the form: [exec|pipe] <command> returns (<return code>, [case] <stdout regexp>) If 'exec' is used, the command is executed. If 'pipe', the mail is piped to the command's stdin. The <command> is a simple string. % tokens are replaced as normal. Any of the <return code> or <stdout regexp> or both may be specified. The <return code> is a simple number which is compared against the return code from the command, the <stdout regexp> is a regexp that is matched case insensitively against each line output by the command on stdout. The result of any bracket expressions in the stdout regexp are saved as 'command0' to 'command9' tags on the mail. Any output on stderr is logged by fdm, so 2>&1 must be included in the command in order to apply the regexp to it. Examples: match exec "true" (0, ) action "act" match not pipe "grep Bob" (1, ) action "act" match pipe "myprogram" (, "failed") actions { "act1" "act2" } match exec "blah" (12, "^Out") action "meep"7.10 Attachments
There are five conditions available to filter based on the size, quantity, type and name of attachments. They are all prefixed with the 'attachment' keyword. Two compare the overall number of attachments: The 'attachment count' conditions matches if the number of attachments is equal to, not equal to, less than or greater than the specified number: match attachment count == 0 action "action" match attachment count != 10 action "action" match attachment count < 2 action "action" match attachment count > 7 action "action" The 'attachment total-size' condition is similar, but compares the total size of all the attachments in a mail: match attachment total-size < 4096 kilobytes action "action" match attachment total-size > 1M action "action" There are also three conditions which matches if any individual attachment fulfils the condition: 'any-size' to match if any attachment is less than or greater than the given size, and 'any-type' and 'any-name' which compare the attachment MIME type and name attribute (if any) using fnmatch(3): match attachment any-size < 2K action "action" match attachment any-type "*/pdf" action "action" match attachment any-name "*.doc" action "action"7.11 Matching tags
The existence of a tag may be tested for using the 'tagged' condition: match tagged "mytag" action "a" match tagged "ohno" and size >1K action drop Or the tags value matched using the 'string' match type (in a similar way to matching bracket expressions): match string "%[group]" to "comp.lang.c" action "clc" match string "%u" to "bob" action "bob"7.12 Using caches
fdm has builtin support for maintaining a cache of string keys, including appending to a cache, checking if a key is present in a cache, and expiring keys from a cache once they reach a certain age. These caches should not be confused with the NNTP cache file. Key caches are referenced by filename and must be declared before use: cache "%h/path/to/cache" cache "~/.fdm.db" expire 1 month If the expiry time is not specified, items are never expired from the cache. Once declared, keys may be added to the cache with the 'add-to-cache' action: match all action add-to-cache "~/my-cache" key "%[message_id]" Or removed with the 'remove-from-cache' action: match all action remove-from-cache "~/my-cache" key "%[message_id]" And the existence of a key in the cache may be tested for using the 'in-cache' condition: match in-cache "~/my-cache" key "%[message_id]" action "foundincache" Any string may be used as key, but the message-id is most often useful. Note that the key may not be empty, so care must be taken with messages without message-id (such as news posts fetched with NNTP). Caches may be used to elimate duplicate messages using rules similar to those above: $db = "~/.fdm-duplicates.db" $key = "%[message_id]" cache $db expire 2 weeks match not string $key to "" { match in-cache $db key $key action maildir "%h/mail/duplicates" match all action add-to-cache $db key $key continue }7.13 Cache commands
fdm includes a number of commands to manipulate caches from the command-line. These are invoked with the 'cache' keyword followed by a command. The following commands are supported: cache add <path> <string> cache remove <path> <string> These add or remove <string> as a key in the cache <path>. cache list [<path>] This lists the number of keys in a cache, or in all caches declared in the configuration file if <path> is omitted. cache dump <path> This dumps the contents of the cache <path> to stdout. Each key is printed followed by a space and the timestamp as Unix time. cache clear <path> Delete all keys from a cache. Examples: $ fdm cache list /export/home/nicholas/.fdm.d/duplicates: 4206 keys $ touch my-cache $ fdm cache dump my-cache $ fdm cache add my-cache test $ fdm cache dump my-cache test 11950724038 Setting options
fdm has a number of options that control its behaviour. These are defined using the set command: set <option> [<value>] In addition to the options below, some environment variables may be used to control fdm. If TMPDIR is present, its value will be used instead of /tmp for saving temporary files. The possible options are: - maximum-size <size> This specifies the maximum size of mail that fdm will accept. The default is 32 MB. Note that fdm may be storing a number of mails simultaneously, up to the 'queue-high' setting (doubled if rewrite is used) for each account, so care should be taken when increasing this option. If mail over the maximum size is encountered, fdm will abort with an error, without deleting the mail from the server (unless 'delete-oversized' is set). - delete-oversized If this option is set, fdm will delete oversized mail from the server rather than leaving it for the user to sort out. - queue-high <number> This sets the number of mails fdm will queue simultaneously. The default is two. Once this limit is reached, fdm will cease fetching mail until the number queued drops to the value of the 'queue-low' setting. fdm queues mails so that they may be processed while waiting for data from the server. Changing this option can increase (or decrease) performance, it's usefulness varies wildly with the ruleset, speed of connection and remote host, and the local hardware. The maximum possible value is currently 50. To ensure fdm fetches and delivers mail sequentially, set this option to one. - queue-low <number> This sets the number of mails fdm will wait for the queue to drop to before restarting fetching after the 'queue-high' limit has been reached. The default is three-quarters of the 'queue-high' setting. - allow-multiple This option makes fdm ignore the lock file, similar to the '-m' command line option. - lock-file <path> This option specifies a string to use as the path of the lock file. For example: set lock-file "/tmp/my-lock-file" - command-user <user> This specifies the user used to run exec and pipe actions. By default it is the user who invoked fdm. - default-user <user> This specifies the default user to use if run as root and no users are specified on the action or filtering rule. This option may be overriden with the '-u' flag on the command line. - lock-types [fcntl] [flock] [dotlock] These specify the type of locking to use when writing to mboxes. fcntl and flock locking are mutually exclusive, but dotlock may be used with either. Some NFS servers do not support fcntl. The default is flock only. Example: set lock-types fcntl dotlock - [domain|domains] <domains> This specifies a domain or list of domains to use when searching for users in mail headers with the 'from-headers' keyword. Examples include: set domain "xyz.ath.cx" set domains { "abc.ca" "def.co.uk" } The default is the hostname, FQDN and IP of the local machine. - [header|headers] <headers> This specifies a header or list of headers to search with the 'from-headers' keyword, such as: set header { "to" "cc" } set headers "x-envelope-to" The default is "to" "cc". - proxy <url> This specifies a URL to proxy outgoing connections through. See the section on proxying below. - unmatched-mail [drop|keep] This option controls how fdm should deal with mail that reaches the end of the ruleset (doesn't match any rules, or only rules with the 'continue' keyword). If 'drop' is specified, such mail is dropped. If 'keep', it is kept. The default is to keep the mail, and issue a warning. - purge-after <message count> The 'purge-after' option instructs fdm to attempt to purge deleted mail after the specified number of mails has been fetched. This is useful to limit the number of mails refetched on the next run if the connection fails. It can have a large effect on performance, particularly if the message count is set to a low number. Note that for POP3, purging deleted mail involves disconnecting and reconnecting from the server; some POP3 servers refuse reconnections if too many are made too quickly, so this option should be used with care. - no-received If this option is present, fdm will not insert a 'Received' header into fetched mail. - no-create When this option is specified, fdm will not attempt to create maildir and mboxes or directories above them. - file-umask [user|<umask>] This specifies the umask to use when creating files. 'user' means to use the umask set when fdm is started, or it may be specified as a three-digit octal number. The default is 077. - file-group [user|<group>] This option allows the default group ownership of files and directories created by fdm to be specified. 'group' may be a group name string or a numeric gid. If 'user' is used, or if this option is not set in the configuration file, fdm does not attempt to set the group of new files and directories. - timeout <time> This controls the maximum time to wait for a server to send data before closing a connection. The default is 900 seconds. - verify-certificates This instructs fdm to verify SSL certificates for all SSL connections, see the previous section on SSL certificate verification.9 Archiving and searching mail
As fdm can fetch from maildirs, it can be used to filter mail from one maildir into another, for example to archive older mail (with drop) or to search for mail matching a particular pattern (with keep). The 'age' condition, and the %[maildir] and time/date tags are particularly useful for archiving. For example, to archive all mail older than 30 days by quarter, something like this may suffice (the account restriction can be dropped if being used in a single-purpose configuration file): account "archive" disabled maildir "source-maildir" match account "archive" and age > 30 days action mbox "%[maildir]q%Q" match account "archive" action keep Then, fdm may be run to move the mail: fdm -vaarchive fetch To search mail, similar rules may be used, but all mail should be kept, in this example by marking the account with 'keep' so that mail is kept no matter what rules it matches: account "search" disabled maildir "source-maildir" keep action "found" mbox "search-results" match "^From.*Bob" in headers account "search" action "found" match account "search" action keep All mail matching the regexp will be copied to the target mbox. There are several other ways to write this ruleset.10 Using fdm behind a proxy
fdm may be used behind a proxy by specifying the proxy URL using the 'proxy' option. HTTP and SOCKS5 proxies are supported: set proxy "http://proxy.server/" set proxy "http://proxy.server:port/" set proxy "socks5://proxy.server/" set proxy "socks5://proxy.server:port/" set proxy "socks5://user@proxy.server/" set proxy "socks5://user:pass@proxy.server/" Authentication is not supported for HTTP proxies.11 Bug reports and queries
Bug reports, queries, suggestions and code are best sent by email to: nicm@users.sf.net Bug reports may also be registered on Sourceforge, but they are likely to be dealt with much faster by email, particularly as it is easier to elicit further information if it is required. A mailing list is available for fdm users, see: https://lists.sourceforge.net/lists/listinfo/fdm-users12 Frequently asked questions
12.1 Why?
Two main reasons: 1. I didn't like the existing tools. That is not to say that other tools are bad or aren't useful, merely that fdm meets my needs better. And more importantly: 2. I disliked the fact that as a home user with a relatively simple setup I had to have five programs to deal with mail: sendmail, fetchmail, procmail, archivemail, mutt; all with different, variously broken configuration file syntaxes, and weird quirks. I now have three programs: sendmail, fdm and mutt, and fewer weird configurations to learn and potential problems. I can also do some quite complex things without reaching for additional tools like formail and reformail.12.2 How do I write regexps?
See the re_format(7) man page. It is online here (for OpenBSD): http://www.openbsd.org/cgi-bin/man.cgi?re_format There are also a number of books and probably websites on the subject.12.3 Keep doesn't work with gmail!
This is because gmail is broken, see: http://pyropus.ca/software/getmail/faq.html#faq-notabug-gmail-bug12.4 Why doesn't fdm run as a daemon?
Because that is what cron is for: it comes as standard with all sensible operating systems, and by using it I avoid a lot of code to deal with signals, configuration reload, extra configuration options and general other crap related to running all the time. Daemonising is for servers, for stuff that needs to run periodically, use cron.12.5 Why does fdm fork child processes when not running as root?
Because one design is much cleaner and easier to work with than two. And it has a negligible practical effect on performance in any case.12.6 Can fdm get rid of that crap in []s some mailing lists add to subjects?
Yes! Rewrite the mail using sed to remove the crap, for example: action "full-disclosure" { rewrite "sed 's/^\\(Subject:.*\\)\\[Full-disclosure\\] /\\1/'" maildir "%h/mail/full-disclosure" } match "^List-Id:.*<full-disclosure\\.lists\\.grok\\.org\\.uk>" in headers action "full-disclosure"12.7 I'm building on Linux and it complains about REG_STARTEND.
Older versions (pre-2005) of glibc don't support the REG_STARTEND flag to regexec(3). Either upgrade to a later version of glibc, or use PCRE.12.8 Should I use PCRE or standard POSIX regexps? What's the difference?
PCRE is a library providing "perl-compatible" regexps. These are broadly compatible with POSIX extended regexps, but with a number of extensions based on perl regexps. Unless you want or need some of the extensions, there is generally no compelling reason to choose one over the other. PCRE is faster than some POSIX regexp implementations, but few rulesets will include sufficient regexps for this to make any difference. The Debian package and FreeBSD port both use PCRE by default. Because I hate maintaining and testing two sets of code, there is a strong possibility that PCRE may become the fdm default as well at some point. ================================================================================ $Id: MANUAL.in,v 1.68 2008/02/10 07:07:07 nicm Exp $
FDM(1) OpenBSD Reference Manual FDM(1)
NAME
fdm - fetch and deliver mail
SYNOPSIS
fdm [-klmnqv] [-a account] [-D name=value] [-f conffile] [-u user]
[-x account] [fetch | poll]
DESCRIPTION
The fdm program fetches mail from a POP3 or IMAP server or from stdin and
delivers it based on a ruleset in the configuration file.
The options are as follows:
-a name
Process only the specified account. This option may appear mul-
tiple times. The account name may include shell glob characters
to match multiple accounts.
-D name=value
This option defines a macro for use when parsing the configura-
tion file. The macro name must be prefixed with $ or % to speci-
fy a string or numeric macro. This option may appear multiple
times.
-f conffile
Specify the configuration file location. Default is ~/.fdm.conf,
or /etc/fdm.conf if that doesn't exist.
-k Keep all mail after delivery, regardless of whether it matches a
drop action. Note that mails kept in this way will be refetched
by fdm if it is run again on the same account.
-l Log using syslog(3) rather than to stderr.
-m Ignore the lock file and run regardless of other instances of
fdm.
-n Do not process any accounts, just verify the configuration file
syntax and exit.
-q Quiet mode. Only print errors.
-u user
Specify the default user for delivery. This overrides the
default-user option in the configuration file.
-v Request verbose logging. This option may be specified multiple
times. -vv will print information on configuration (useful with
-n). -vvvv duplicates all traffic to and from remote servers to
stdout. This feature is disabled when using the -l flag.
-x name
Exclude the named account. Multiple -x options may be specified.
As with -a, shell glob characters may be used.
fetch | poll | cache
The fetch command instructs fdm to fetch and deliver messages.
The poll command polls the accounts in the configuration file and
reports a message count for each. cache allows fdm cache files
to be manipulated: see the next section.
CACHE COMMANDS
The following cache manipulation commands are supported:
cache add path string
cache remove path string
Add or remove string as a key in the cache at path.
cache list [path]
List the number of keys in the specified cache, or if path is
omitted, in all caches declared in the configuration file.
cache dump path
Dump the contents of the cache path to stdout. Each key is
printed followed by a space and the timestamp as Unix time.
cache clear path
Delete all keys from the cache at path.
FILES
~/.fdm.conf
default fdm configuration file
/etc/fdm.conf
default system-wide configuration file
~/.fdm.lock
default lock file
/var/db/fdm.lock
lock file for root user
SEE ALSO
mail(1), fdm.conf(5), sendmail(8)
AUTHORS
Nicholas Marriott <nicm@users.sourceforge.net>
OpenBSD 4.2 August 14, 2006 2
FDM.CONF(5) OpenBSD Programmer's Manual FDM.CONF(5)
NAME
fdm.conf - fdm configuration file
DESCRIPTION
This manual page describes the fdm(1) configuration file. It defines
accounts from which to fetch mail, a number of possible actions to take,
and rules connecting a regexp with an action. The file is parsed once
from top to bottom, so action and account definitions must appear before
they are referenced in a rule. Rules are evaluated from first to last
and (unless overridden by the continue keyword) evaluation stops at the
first match.
The file has the following format:
Empty lines and lines beginning with the `#' character are ignored.
Regexps and strings must be enclosed in double quotes. Special charac-
ters in regexps and strings (including passwords) must be escaped. Note
that this may mean double-escaping in regexps.
Possible commands are covered in the following sections.
OPTIONS
Options are configured using the set command. It may be followed by the
following options, one per command:
maximum-size size
This is used to set the maximum size of a mail. Mails larger
than this limit are dropped and, if applicable, not deleted from
the server.
The size may be specified as a plain number in bytes or with a
suffix of `K' for kilobytes, `M' for megabytes or `G' for giga-
bytes. The default is one gigabyte.
delete-oversized
If this option is specified, fdm(1) attempts to delete messages
which exceed maximum-size, and continue. If it is not specified,
oversize messages are a fatal error and cause fdm(1) to abort.
Note that fdm(1) may have a number of messages queued (up to the
queue-high setting, doubled for rewrite, per account), so this
setting and the queue-high option should be set after considera-
tion of the space available in the temporary folder and the im-
plications should fdm(1) abort due to the space becoming full.
queue-high number
This sets the maximum number of messages fdm(1) will hold simul-
taneously. fdm(1) will attempt to process previously queued mes-
sages as the next is being fetched. Once this limit is reached,
no further messages wil be fetched until the number of messages
held drops to the queue-low value.
queue-low number
This is the length to which the message queue must drop before
fetching continues after the queue-high limit has been reached.
allow-multiple
If this option is specified, fdm(1) does not attempt to create a
lock file and allows multiple instances to run simultaneously.
lock-file path
This sets an alternative lock file. The default is ~/.fdm.lock
for non-root users and /var/db/fdm.lock for root.
command-user user
This specifies the user used to run exec and pipe actions. By
default it is the user who invoked fdm.
default-user user
This sets the default user to change to before delivering mail,
if fdm(1) is running as root and no alternative user is specified
as part of the action or rule. This option may be overridden
with the -u switch on the command line. A default user must be
given if running as root.
lock-types type ...
This specifies the locks to be used for mbox locking. Possible
types are fcntl, flock, and dotlock. The flock and fcntl types
are mutually exclusive. The default is flock.
domain domain | domains { domain ... }
This specifies the domains to be used when looking for users with
the from-headers keyword. The default is the computer's host-
name.
header header | headers { header ... }
This allows the headers to be examined when looking for users to
be set. The default is to look only at the "From" and "Cc" head-
ers. The headers are case-insensitive.
proxy url
This instructs fdm(1) to proxy all connections through url. HTTP
and SOCKS5 proxies are supported at present (URLs of the form
http://host[:port] or socks://[user:pass@]host[:port]). No au-
thentication is supported for HTTP.
unmatched-mail drop | keep
This option controls what fdm(1) does with mail that reaches the
end of the ruleset (mail that matches no rules or matches only
rules with the continue keyword). drop will cause such mail to
be discarded, and keep will attempt to leave the mail on the
server. The default is to keep the mail and log a warning that
it reached the end of the ruleset.
purge-after count
The purge-after option makes fdm(1) attempt to purge deleted mail
from the server (if supported) after count mails have been re-
trieved. This is useful on unreliable connections to limit the
potential number of mails refetched if the connetion drops, but
note that it can incur a considerable speed penalty.
no-received
If this option is present, fdm(1) will not insert a `Received'
header into each mail.
no-create
If this option is set, fdm(1) will not attempt to create maildirs
and mboxes or missing elements of their paths.
file-umask user | umask
This specifies the umask(2) to use when creating files. user
means to use the umask set when fdm(1) is started, or umask may
be specified as a three-digit octal number. The default is 077.
file-group user | group
This option allows the default group ownership of files and di-
rectories created by fdm(1) to be specified. group may be a
group name string or a numeric gid. If user is used, or this op-
tion does not appear in the configuration file, fdm(1) does not
attempt to set the group of new files and directories.
timeout time
This controls the maximum time to wait for a server to send data
before closing a connection. The default is 900 seconds.
verify-certificates
Instructs fdm(1) to verify SSL certificates for all SSL connec-
tions.
INCLUDING FILES
Further configuration files may be including using the include command:
include path
The file to include is searched for first as an absolute path and then
relative to the directory containing the main configuration file.
MACROS
Macros may be defined using the following syntax:
$name = string
%name = number
Macros are prefixed with $ to indicate a string value and % to indicate a
numeric value. Once defined, a macro may be used in any place a string
or number is expected. Macros may be embedded in strings by surrounding
their name (after the $ or %) with {}s, like so:
"abc ${mymacro} %{anothermacro} def"
The ifdef, ifndef and endif keywords may be used to conditionally parse a
section of the configuration file depending on whether or not the macro
given exists or does not exist. ifdef and ifndef blocks may be nested.
SHELL COMMANDS
The result of a shell command may be used at any point a string or number
is expected by wrapping it in $() or %(). If the former is used, the
command result is used as a string; if the latter, it is converted to an
integer. Shell commands are executed when the configuration file is
parsed.
ACCOUNTS
The account command is used to instruct fdm(1) to fetch mail from an ac-
count. The syntax is:
account name [users] [disabled] type [args] [keep]
The name argument is a string specifying a name for the account. The op-
tional users argument has the following form:
user user | users { user ... } | user from-headers
The first two options specify a user or list of users as which the mail
should be delivered when an action is executed. If user from-headers is
specified, fdm(1) attempts to find the users from the mail headers, using
the values of the headers and domains options. If no headers are speci-
fied, or fdm(1) fails to find any valid users in the headers, the default
user (set with set default-user) is used. Users specified as part of the
account definition may be overridden by similar arguments to action defi-
nitions or on match rules. If fdm(1) is run as non-root, it will still
execute any actions once for each user, but will be unable to change to
that user so the action will be executed multiple times as the current
user.
The disabled keyword instructs fdm(1) to ignore this account unless it is
explicitly enabled with a -a option on the command line. If the keep
keyword is specified, all mail collected from this account is kept (not
deleted) even if it matches a drop action.
Supported account types and arguments are:
stdin This account type reads mail from stdin, if it is connected to a
pipe. This may be used to deliver mail from sendmail(8), see
fdm(1) for details.
pop3 server host [port port] [user user] [pass pass] [only] [no-apop]
pop3s server host [port port] [userpass] [only] [no-apop] [no-verify]
These statements define a POP3 or POP3S account. The userpass
element has the following form:
[user user] [pass pass]
The host, user and pass arguments must be strings. If the user
or pass is not provided, fdm(1) attempts to look it up in the
~/.netrc file (see ftp(1) for details of the file format). The
port option may be either a string which will be looked up in the
services(5) database, or a number. If it is omitted, the default
port (110 for POP3, 995 for POP3S) is used.
The only option takes the form:
[new-only | old-only] cache path
new-only fetches only mail not previously fetched, and old-only
is the inverse: it fetches only mail that has been fetched be-
fore. The cache file is used to save the state of the POP3 mail-
box. The no-apop flag forces fdm(1) not to use the POP3 APOP
command for authentication, and the no-verify keyword instructs
fdm(1) to skip SSL certificate validation for this account.
pop3 pipe command [userpass] [only] [no-apop]
This account type uses the POP3 protocol piped through command,
such as ssh(1). If the command produces any output to stderr, it
is logged. For POP3 over a pipe, providing a user and password
is not optional and it may not be read from ~/.netrc.
imap server host [port port] [userpass] [folder name] [only]
imaps server host [port port] [userpass] [folder name] [only] [no-verify]
These define an IMAP or IMAPS account. The parameters are as for
a POP3 or POP3S account, aside from the additional folder option
which allows the folder name to be specified (the default is to
fetch from the inbox). The default ports used are 143 for IMAP
and 993 for IMAPS. For IMAP, the only item consists only of one
of the keywords new-only or old-only - a cache file is not re-
quired.
imap pipe command [userpass] [folder name] [only]
As with pop3 pipe, this account type uses the IMAP protocol piped
through command. If the optional IMAP user and pass are sup-
plied, they will be used if necessary, but if one is provided,
both must be - using ~/.netrc is not permitted.
maildir path
maildirs { path ... }
These account types instruct fdm(1) to fetch mail from the
maildir or maildirs specified. This allows fdm(1) to be used to
filter mail, fetching from a maildir and deleting (dropping) un-
wanted mail, or delivering mail to another maildir or to an mbox.
mbox path
mboxes { path ... }
These are similar to maildir and maildirs, but cause fdm(1) to
fetch mail from an mbox or set of mboxes.
nntp server host [port port] group group cache cache
nntp server host [port port] groups { group ... } cache cache
nntps server host [port port] group group cache cache
nntps server host [port port] groups { group ... } cache cache
An NNTP account. Articles are fetched from the specified group
or groups and delivered. The index and message-id of the last
article fetched in each group is saved in the specified cache
file. When fdm(1) is run again, fetching begins at the cached
article.
Note that the keep option is completely ignored for NNTP accounts - all
mail is kept, and the cache is always updated.
TAGGING
As mail is processed by fdm(1), it is tagged with a number of name/value
pairs. Some tags are added automatically, and mail may also be tagged
explicitly by the user using the tag action. Tags may be inserted in
most strings in a similar manner to macros, except tags are processed at
runtime rather than as the configuration file is parsed. A tag's value
is inserted by wrapping its name in %[], for example:
abc%[account]def
%[hour]:%[minute]:%[second]
The default tags also have a single-letter shorthand. Including a nonex-
istent tag in a string is equivalent to including a tag with an empty
value, so "abc%[nonexistent]def" will be translated to "abcdef".
The automatically added tags are:
account (%a)
The name of the account from which the mail was fetched.
home (%h)
The delivery user's home directory.
uid (%n)
The delivery user's uid.
action (%t)
The name of the last action executed for this mail.
user (%u)
The delivery user's username.
hour (%H)
The current hour (00-23).
minute (%M)
The current minute (00-59).
second (%S)
The current second (00-59).
day (%d)
The current day of the month (01-31).
month (%m)
The current month (01-12).
year (%y)
The current year.
year2 The current year as two digits.
dayofweek (%W)
The current day of the week (0-6, Sunday is 0).
dayofyear (%Y)
The current day of the year (001-366).
quarter (%Q)
The current quarter (1-4).
rfc822date
The current date in RFC822 format.
mail_hour
The hour from the mail's date header, if it exists and is
valid, otherwise the current time.
mail_minute
The minute from the mail's date header.
mail_second
The second from the mail's date header.
mail_day
The day from the mail's date header.
mail_month
The month from the mail's date header.
mail_year
The year from the mail's date header.
mail_year2
The same as two digits.
mail_rfc822date
The mail's date in RFC822 format.
hostname
The local hostname.
In addition, the shorthand %% is replaced with a literal %, and %0 to %9
are replaced with the result of any bracket expressions in the last reg-
exp.
CACHES
fdm(1) can maintain a cache file with a set of user-defined strings. In
order to use caches, fdm(1) must have been compiled with them enabled.
Caches are declared with the cache keyword:
cache path [expire age]
The path is the location of the cache file. If the expire keyword is
specified, items in the cache are removed after they reach the age speci-
fied. age may be given unadorned in seconds, or followed by one of the
modifiers: seconds, hours, minutes, days, months or years.
Caches must be declared before they are used. Items are added to caches
using the add-to-cache action, removed using the remove-from-cache ac-
tion, and searched for using the in-cache condition; see below for infor-
mation on these.
ACTIONS
The action command is used to define actions. These may be specified by
name in rules (see below) to perform some action on a mail. The syntax
is:
action name [users] action
action name [users] { action ... }
The name is a string defining a name for the action. The users argument
has the same form as for an account definition. An action's user setting
may be overridden in the matching rule.
The possible values for action are listed below. If multiple actions are
specified they are executed once in the order specified, for each user.
drop Discard the mail.
keep Keep the mail, do not remove it from the account.
tag string [value value]
This tags mail with string, and optionally value, which may be
matched using the tagged or string conditions.
maildir path
Save the mail to the maildir specified by path. If the maildir
or any part of its path does not exist, it is created, unless the
no-create option is set.
Mail delivered to a maildir is tagged with a mail_file tag con-
taining the full path of the mail file.
mbox path [compress]
Append the mail to the mbox at path. If compress is specified,
fdm(1) will add `.gz' to path and attempt to write mail using
gzip(1) compression. If the mbox or any part of its path does
not exist, it is created, unless the no-create option is set.
Mail delivered to an mbox is tagged with a mbox_file tag contain-
ing the path of the mbox.
exec command
Execute command.
pipe command
Pipe the mail to command. exec and pipe commands are run as the
command user.
write path
Write the mail to path.
append path
Append the mail to path.
smtp server host [port port] [from from] [to to]
Connect to an SMTP server and attempt to deliver the mail to it.
If from or to is specified, they are passed to the server in the
MAIL FROM or RCPT TO commands. If not, the current user and host
names are used.
rewrite command
Pipe the entire mail through command to generate a new mail and
use that mail for any following actions or rules. An example of
the rewrite action is:
action "cat" pipe "cat"
action "rewrite" rewrite "sed 's/bob/fred/g'"
# this rule will rewrite the message
match all action "rewrite" continue
# this rule will cat the rewritten message
match all action "cat"
add-header name value value
Add a header name with contents value.
remove-header name
remove-headers { name ... }
Remove all occurances of headers matching the fnmatch(3) pattern
name.
stdout Write the mail to stdout.
add-to-cache path key key
This action adds the string key to the cache specified by path.
If key already exists in the cache, it is replaced.
remove-from-cache path key key
Remove the string key from the cache path, if a matching key is
present.
action name
This invokes another named action. A maximum of five actions may
be called in a sequence.
RULES
Rules are specified using the match keyword. It has the following basic
form:
match condition [and | or condition ...] [users] actions [continue]
The condition argument may be one of:
all Matches all mail.
matched
Matches only mail that has matched a previous rule and been
passed on with continue.
unmatched
The opposite of matched: matches only mails which have matched no
previous rules.
account name | accounts { name ... }
Matches only mail fetched from the named account or accounts.
The account names may include shell glob wildcards to match mul-
tiple accounts, as with the -a and -x command line options.
tagged string
Matches mails tagged with string.
[case] regexp [in headers | in body]
Specifies a regexp against which each mail should be matched.
The regexp matches may be restricted to either the headers or
body of the message by specifying either in headers or in body.
The case keyword forces the regexp to be matched case-sensitive-
ly: the default is case-insensitive matching.
exec command [user user] returns (return code, stdout regexp)
pipe command [user user] returns (return code, [case] stdout regexp)
These two conditions execute a command and test its return value
and output. The return code argument is the numeric return code
expected and stdout regexp is a regexp to be tested against the
output of the command to stdout. Either of these two arguments
may be omitted: if both are specified, both must match for the
condition to be true. The pipe version will pipe the mail to the
command's stdin when executing it. If a user is specified,
fdm(1) will change to that user before executing the command,
otherwise the current user (or root if started as root) is used.
size < number
size > number
Compare the mail size with number.
string string to [case] regexp
Match string against regexp.
age < time
age > time
The age condition examines the mail's date header to determine
its age, and matches if the mail is older (>) or newer (<) than
the time specified. The time may be given as a simple number in
seconds, or followed by the word seconds, hours, minutes, days,
months or years to specify a time in different units.
in-cache path key key
This condition evaluates to true if the string key is in the
cache at path.
attachment count < number
attachment count > number
attachment count == number
attachment count != number
These conditions match if the mail possesses a number of attach-
ments less than, greater than, equal to or not equal to number.
attachment total-size < size
attachment total-size > size
Matches if the total size of all attachments is smaller or larger
than size.
attachment any-size < size
attachment any-size > size
Compare each individual attachment on a mail to size and match if
any of them are smaller or larger.
attachment any-type string
attachment any-name string
Match true if any of a mail's attachments possesses a MIME type
or filename that matches string. fnmatch(3) wildcards may be
used.
Multiple conditions may be chained together using the and or or keywords.
The conditions are tested from left to right. Any condition may be pre-
fixed by the not keyword to invert it.
The optional users argument to the first form has the same syntax as for
an action definition. A rule's user list overrides any users given as
part of the actions.
The actions list specifies the actions to perform when the rule matches a
mail. It is either of a similar form:
action name | actions { name ... }
Or may specify a number of actions inline (lambda actions):
action action
action { action ... }
In the latter case, action follows the same form as described in the AC-
TIONS section. The actions are performed from first to last in the order
they are specified in the rule definition.
If the continue keyword is present, evaluation will not stop if this rule
is matched. Instead, fdm(1) will continue to match further rules after
performing any actions for this rule.
NESTED RULES
Rules may be nested by specifying further rules in braces:
match condition [and | or condition ...] {
match ...
}
The inner rules will not be evaluated unless the outer one matches.
Rules may be multiply nested. Note that the outer rule does not count as
a match for the purposes of the matched and unmatched conditions.
FILES
~/.fdm.conf default fdm.conf configuration file
/etc/fdm.conf default system-wide configuration file
~/.fdm.lock default lock file
/var/db/fdm.lock lock file for root user
SEE ALSO
fdm(1), re_format(7)
AUTHORS
Nicholas Marriott <nicm@users.sourceforge.net>
OpenBSD 4.2 August 21, 2006 10