=head1 NAME

News::Gateway - Tools for gatewaying messages between news and mail.

=head1 SYNOPSIS

    use News::Gateway;
    my $gateway = News::Gateway->new (1, 'user@example.com');
    $gateway->modules ('headers', mailtonews => ['misc.test']);
    $gateway->config_file ('/path/to/config/file');
    $gateway->read (\*STDIN);
    my $error = $gateway->apply ();
    if ($error) { $gateway->error ($error) }
    $error = $gateway->post ();
    if ($error) { $gateway->error ($error) }

=head1 DESCRIPTION

This module is a collection of tools to allow gatewaying of mail messages
into newsgroups or vice versa.  It's written to be extremely modular and
extendable, so that any modification or check of incoming messages can be
dropped in.  It's intended to be usable for both a straight gateway or a
robomoderator, or even as the front end and supporting tool set for human
moderation.

The module is intended to take input from three different sources:  an
incoming message, a set of modules that contain checks or modifications to
be performed to that message, and a configuration file that affects the
behavior of the modules.  Each module can list one or more configuration
directives that it takes, and then when those directives are encountered
in the configuration file, they're parsed and and handed off to the
module.  Then, after the message has been read in, its headers and body
are given to each module in turn, so that they can perform whatever tests
and make whatever modifications they were designed to do.

A message is represented internally as a News::Article object that each
module has access to, allowing the headers or body to be read and
manipulated.

=head1 CONSTRUCTOR

=over 4

=item new (INTERACTIVE [, MAINTAINER [, SENDER]])

This builds a new News::Gateway object.  C<INTERACTIVE> is a true/false
flag specifying whether the program is running interactively.  If true,
the C<error()> method will call C<die> in the event of an error;
otherwise, it will attempt to send mail to the maintainer and the sender
of the message.  C<MAINTAINER> is the e-mail address of the maintainer of
the gateway, defaulting to the user the program is running as if not
supplied.  C<SENDER> is the envelope sender which should be used for all
mail sent through News::Gateway, defaulting to C<MAINTAINER> if not
supplied.

=back

=head1 METHODS

First, the standard methods in the order in which they are normally
called.  Some of these methods, like C<modules()>, are normally called
only once, while others, like C<read()>, may be called multiple times for
multiple incoming messages.  You'll almost certainly also want to read
about the C<config()>, C<config_file()>, and C<post()> directives below in
L<"UTILITY MODULES">.

=over 4

=item modules (MODULE [ => ARGUMENTS ] [, ...])

Sets the list of modules that should be applied to each incoming message.
The list of modules is cumulative; in other words, C<modules()> can be
called multiple times to add more modules to the list of ones which will
be applied.  This method both adds each supplied module name to the list
of modules that will be run if C<apply()> is called with no arguments and
sets up callbacks so that any configuration directives that C<MODULE> is
interested in will be passed along to it when configuration directives are
processed.

Some modules may take arguments; if so, they can be passed in as an
anonymous array associated with the module.  For example:

    $gateway->modules (mailtonews => ['misc.test']);

adds C<mailtonews> to the list of modules and passes in C<misc.test> as an
argument to it.

Modules will be run in the order they are given to C<modules()>.

C<modules()> should always be called before any call to C<apply()>,
C<config()>, or C<config_file()> that those modules are supposed to
affect.

=item read (SOURCE [, MAXSIZE [, MAXHEAD]])

Reads in a new message from the supplied source, which can be a file name,
a file handle, or even an anonymous sub which will be called repeatedly
for each line of the message until it returns undef.  C<MAXSIZE>, if
given, specifies the maximum size in bytes of the article body, and
C<MAXHEAD>, if given, specifies the maximum size in bytes of the article
headers.  C<MAXSIZE> defaults to 256KB and C<MAXHEAD> defaults to 8KB.

This method returns the total number of bytes read if successful, or undef
in the case of an error.

This method constructs a new News::Article object and calls its method, so
it will always support essentially the same arguments as the corresponding
News::Article C<read()> method.

=item apply (MODULES)

This method hands the message off to each module given (if no modules are
given, the message is given to each module registered by C<modules()> in
the order in which they were given to C<modules()>) and lets them do
whatever they wish with the article.  If a module returns undef, it's
presumed to be successful and the message is given to the next module.  If
a module returns a string, it's considered to be a failure message and
C<apply()> stops.

If all modules succeed, C<apply()> returns undef or the empty list.  If a
module fails, C<apply()> returns a list consisting of the module which
failed and the message it failed with in a list context; or a string
consisting of the name of the module, a colon, a space, and the message it
failed with in a scalar context.

The calling script should then look at the failure message and take the
appropriate action.

=back

The following additional methods are also supplied, some of which are
mostly intended for internal use.

=over 4

=item error (ERROR)

Handles a fatal error.  If the C<INTERACTIVE> flag passed to C<new()> was
true, this method just calls C<die> with the error message.  Otherwise, it
calls C<mail_error()> (part of the standard C<mail> module), which
attempts to mail the maintainer and the sender of the message.  If that
fails, for whatever reason, News::Gateway calls C<exit> with the right
status code for a mail bounce.

This method should be used B<only> for fatal, unrecoverable errors,
particularly when C<INTERACTIVE> isn't true, since it could bounce mail or
notify the wrong person of problems.  It's a last resort that goes to
great lengths to try to keep the message from being lost.

=item get_article ()

An accessor method that returns the underlying News::Article object.

=item set_article (ARTICLE)

Takes a News::Article object and makes that the underlying message in that
instance of News::Gateway, discarding the old one if any.

=back

=head1 REWRITE MODULES

The following modules to perform checks or rewrites of messages are part
of the standard distribution.  To use them with a News::Gateway object,
pass them (along with anonymous arrays for their options, if any) to the
C<modules()> method.

=head2 anykeyword

This module requires that every incoming message have at least one keyword
on the subject line.  For the purposes of this check, a keyword is defined
as anything that matches:

    /^(?:Re:\s+)?\[\S+\]/

(in other words, something without spaces enclosed in square brackets at
the beginning of the subject line modulo any Re:).  If no keyword is found
in the incoming message, this module returns the message:

    No keyword found

This module takes no arguments or configuration directives.

=head2 bodyheaders

Extracts headers from the beginning of a message body and lifts them up
into the message headers.  This is to assist posters whose software makes
it difficult for them to edit the headers of their posts or mail.

This module looks for recognized headers at the beginning of the article
body.  As long as it's seen only blank lines (possibly containing
whitespace) and recognized headers, it adds all of those headers to the
main message headers.  As soon as it finds a line that isn't a recognized
header, it stops, and if it found any headers it removes everything up to
the first non-header line from the body.

This module takes one configuration directive:

=over 4

=item bodyheaders HEADER [HEADER ...]

A list of headers to look for in the beginning of the body.  The headers
are not case sensitive, and any amount of whitespace (including none) is
allowed after the colon.

=back

For example, with the directive:

    bodyheaders x-no-archive

and a message beginning with:

    x-no-archive:yes

    Comment: hello

an X-No-Archive header with content "yes" will be added to the message
headers and the first line of the message body after this module runs will
be the line starting with "Comment:".  Had "comment" also been listed in
the configuration directive, that header would have been lifted into the
article headers as well and this module would have continued looking.

Whitespace on the line before the header is not allowed, and such a line
would not be recognized as a header to be lifted to the message headers.

This module takes no arguments and cannot fail.

=head2 cleanbody

This module checks the body of an incoming article and cleans it up,
attempting to enforce standard ISO 8859-1 text.  If it's encoded in
quoted-printable, that encoding is undone.  Microsoft Word SmartQuotes are
converted into their ASCII equivalents, any Ctrl-Ms are removed, and the
module checks to make sure that the message contains no invalid characters
(outside of the standard ISO 8859-1 range) and that no lines are longer
than 79 characters.

The messages returned by the checks in the event of failure are:

=over 4

=item Invalid characters in body

The body contains characters outside the range [\s!-~\xa0-\xff].

=item Line over 79 characters

One or more lines in the body are over 79 characters long.

=back

This module takes no arguments or configuration directives.  If the
original message was in quoted-printable, it will change the
Content-Transfer-Encoding header to be "8bit".  It recognizes
quoted-printable from the MIME encoding headers.

=head2 crosspost

Limits crossposting and followups.  This module takes, in some ways, the
opposite approach from the newsgroups module; rather than listing
newsgroups which are allowed, it allows one to set a limit on the number
of groups crossposted, on the number of groups to which followups would
go, and on which groups can be crossposted to or to which followups can be
directed.

A group can be excluded from the Newsgroups header or from the Followup-To
header in one of two ways; either messages posted to that group or
messages that direct followups to a set of newsgroups including that group
can be rejected, or that group can be silently removed from the Newsgroups
and/or Followup-To headers.  The latter behavior is generally not
recommended for robomoderators, since moderators are generally discouraged
to make decisions about where posts should be posted for the poster.
Returning the post and asking the poster to choose a more appropriate set
of groups is generally a better approach.

This module will remove a Followup-To header (and not add one) if its
content is/would be identical to the Newsgroups header.

This module takes the following configuration directives:

=over 4

=item crosspost max COUNT

Reject any message crossposted to more than C<COUNT> newsgroups.

=item crosspost remove|reject GROUP

Either remove C<GROUP> silently from the Newsgroups header if present or
reject all articles crossposted to C<GROUP>.

=item followup max COUNT

Reject any messages that would direct followups to more than C<COUNT>
newsgroups, bearing in mind that if no Followup-To header is present,
followups would go to all groups in the Newsgroups header.

=item followup remove|reject GROUP

Either remove C<GROUP> silently from the followups if present or reject
all articles crossposted to C<GROUP>.

=back

This module may return one of the following failure messages:

=over 4

=item Invalid crosspost to %s

The message was crossposted to a group mentioned in a C<crosspost reject>
directive.

=item Followups would go to %s

Followups to the message would go to a group mentioned in a C<followup
reject> directive.

=item Excessively crossposted

The message was crossposted to more groups than allowed by a C<crosspost
max> directive.

=item Followups would go to too many groups

Followups to the message would go to more groups than allowed by a
C<followup max> directive.

=back

This module works with a Newsgroups header, so if other modules need to be
applied to construct the Newsgroups header, they should run before this
one.  This module takes no arguments.

=head2 headers

This module implements some general header rewriting functions, including
adding, dropping, and renaming headers and replacing header contents.  It
takes one configuration directive:

=over 4

=item header HEADER FUNCTION [ CONTENT ]

Specify an action on the header C<HEADER>.  C<FUNCTION> can be any of the
following:  drop deletes a header, rename renames the original header to
X-Original-HEADER retaining the same value, ifempty adds a header with
content C<CONTENT> if and only if the message doesn't already contain a
header C<HEADER>, replace replaces all existing C<HEADER> headers with one
containing C<CONTENT>, prepend adds C<CONTENT> to the beginning of the
first header C<HEADER> or creates a new header C<HEADER> with content
C<CONTENT> if none already exists, and reject returns an error if
C<HEADER> is present in the incoming message.

C<CONTENT> can contain various special variables:  C<$n> will be replaced
with the name of the running program, C<$v> will be replaced with the
version of News::Gateway, and C<$i> will be replaced with a unique
identifier formed from the current time and the process ID.  C<$$> will be
replaced with C<$>, so to put a literal dollar sign in a header, you
should use C<$$>.

=back

For example, suppose you have a configuration file with the following
directives:

    header organization add     SNAP
    header message-id   rename
    header sender       drop
    header comment      replace $n $v
    header subject      ifempty no subject (thread id $i)

and suppose you have an incoming message with the headers:

    Organization: Restaurant Reviews
    Message-ID: <123142@bar.org>
    Sender: foo@bar.org
    Comment: Hello
    Comment: Hello again

After the headers module runs, the message will have a header of:

    Organization: Restaurant Reviews
    Organization: SNAP
    X-Original-Message-ID: <123142@bar.org>
    Comment: PROGRAM VERSION
    Subject: no subject (thread id ID)

where C<PROGRAM> is the name of the running program (ie, C<$0>),
C<VERSION> is the version of News::Gateway, and C<ID> is a unique
identifier as described above.

This module may fail and call error() with the following message while
reading the configuration directives:

=over 4

=item Unknown header rewrite action %s

A rewrite action was specified that isn't among those that are supported.
This probably indicates a typo.

=back

This module may fail in one way:

=over 4

=item Invalid header %s

A header that was associated with a reject action in a configuration
directive was present in the incoming message.  Note that the header will
be given in all lowercase.

=back

As a side note, if you're constructing a robomoderator for a newsgroup,
dropping or renaming the Path header in incoming messages is highly
recommended.  It turns out that some news servers will add a Path header
with their hostname B<before> remailing the message to a moderator, and if
you keep that Path header when you post, the article will never propagate
back to the site of the original poster.

=head2 keywords

This module checks the Subject header of a message and ensures that it
starts (modulo an initial Re:) with a valid keyword.  By default, the
following keyword formats are supported:

    KEYWORD:
    KEYWORD/KEYWORD:
    [KEYWORD]
    [KEYWORD/KEYWORD]
    [KEYWORD][KEYWORD]

If multiple keywords are given, they all have to be valid.  Keywords are
checked against a list given in a file and are case-insensitive.

This module takes one optional argument, a reference to a sub which, if
given a subject line, returns a list of all the keywords present.  This
can be used to override the default patterns listed above.

There is one required configuration file directive:

=over 4

=item keywords FILENAME

Specifies the file from which the list of valid keywords will be read.
The file should list all valid keywords, one per line.

=back

Two failure messages are possible:

=over 4

=item No keywords found

There were no keywords in the subject header of the message.

=item Invalid keyword '%s'

A keyword was found that wasn't in the list of valid keywords.

=back

=head2 mailpath

Generates an X-Mail-Path header from the Received headers of a mail
message.  It attempts to figure out the real name of each host that the
mail passed through based on the standard comment syntax used by most
mailers, but this problem is inherently impossible to solve completely.

X-Mail-Path is set to a !-separated list of hosts that the mail has passed
through.  If from the Received headers the name of a host appears to be
trusted, it's given; otherwise the name followed by the IP address in
brackets is given.  If the IP address isn't available, "[UNTRUSTED]" is
appended.

The envelope sender is added to the end of the X-Mail-Path if known; if
not, "UNKNOWN" is added instead.

mailpath takes no arguments or configuration directives and cannot fail.

=head2 mailtonews

Performs a variety of rewrites, changes, and checks necessary to do the
basic conversion of a mail message into a news message.  Most of this
involves deleting or renaming headers that a news server will not accept
or would overwrite and making sure that the required news headers are
present.

This module does the following:  Checks to ensure there is a From header;
compacts the From header down to one line if it contained any continuation
lines since INN can't deal with continuation lines in a From header; adds
a Newsgroups header with the default group if none is present, or checks
the Newsgroups header for validity and removes duplicate groups if one is
present; drop the headers Lines, Received, Relay-Version, and Xref which
will be rejected or overwritten by the news server; renames the headers
NNTP-Posting-Host and Sender to be prefixed with X-Original- since new
ones will be generated; adds a new Sender header pointing to the
maintainer (set in the News::Gateway constructor); renames the Message-ID
header to X-Original-Message-ID if the message ID is syntactically invalid
for a news message (this will cause a new message ID to be generated by
the server); checks the In-Reply-To header if present to see if it
contains something that looks like a message ID and if so adds it to the
end of the References header if and only if it doesn't duplicate the
message ID on the end of the References header; and sets the subject of
the article to the value "(none)" if there is no Subject header.

This module takes one argument, the default newsgroup to post to.  A
Newsgroups header containing this group will be added if no Newsgroups
header is present in the incoming message.  This argument is optional, but
if you're gating mail messages to news messages you'll pretty much always
want to supply it.  While it's tempting to assume that all submissions to
a moderated group will already have a Newsgroups header added by the news
server, you'll find in practice that people will often mail submissions
directly to the submission address, or that news servers won't handle
things in the way that you would expect.

This module (possibly combined with the headers module) may be sufficient
for generating the Newsgroups header if the program only handles one
newsgroup, but any more complicated situation will probably require that a
module (such as the newsgroups module) be run before this one to generate
the Newsgroups header.

There are three possible failure messages which may be returned;

=over 4

=item Empty body

The body of the message was empty.  Most news servers will reject messages
with an empty body.

=item Missing required From header

The incoming message doesn't have a From header.  A From header is
required for all messages.

=item No Newsgroups header or default group

The incoming message didn't have a Newsgroups header and no argument was
supplied to the module.  If you don't supply an argument specifying the
default newsgroup, all incoming messages must have a Newsgroups header
before this module is run, since a Newsgroups header is mandatory in a
news article.

=back

This module takes no configuration directives.

=head2 moosesign

Does simple PGPMoose signing of messages.  This module is B<not> a full
implementation of PGPMoose; in particular, it doesn't support all of the
crosspost handling.  It looks through the groups in the Newsgroups header
and adds a PGPMoose X-Auth header for every group in the Newsgroups header
for which a PGP key has been given.

This module takes one configuration directive, which associates a PGP key
ID and passphrase with a newsgroup:

=over 4

=item pgpkey NEWSGROUP PASSPHRASE KEYID

Associates the given C<KEYID> and C<PASSPHRASE> with C<NEWSGROUP>.
C<KEYID> is optional and will default to the name of the newsgroup
surrounded by spaces if not given.  (This default will allow PGP to find
the key correctly if the key ID is something like "Moderator for
group.name <address>" while avoiding confusing group.name and
group.name.other.)

=back

Note that using this module means putting the PGP passphrases for the
moderation keys in cleartext in the configuration file or in the script.

This module takes no arguments and cannot fail.

=head2 mungeids

Programmatically munges the message IDs in the Message-ID and References
headers by adding a slash-separated list of newsgroups to which the
article is going to be posted, followed by a slash, to the beginning of
them.  We also strip off any prefix that looks like a prefix we'd generate
(any number of strings that could be a newsgroup name, separated by
slashes) to prevent threads with replies from readers of multiple
mungeid'd groups from generating constantly-growing message IDs and to
hopefully still get threading right.  For the purposes of this check, we
assume newsgroup names will contain at least one period; this module may
not work correctly for newsgroup names that do not.

The purpose of this is for use with mailing list to newsgroup gateways.
Since messages to multiple mailing lists may all have the same message ID,
and since the same mailing list may be gated to Usenet in multiple places,
the message IDs of incoming mailing list messages need to be munged in
some fashion before being passed on to Usenet.  Ideally, however,
threading should be preserved, and replies to mailing list messages from
other people on the mailing list won't refer to the munged message ID.
This module therefore applies the same munge to all of the message IDs in
the References header as well, in the hope of maintaining threading and
still getting message ID uniqueness.

If the message doesn't already have a Message-ID header, this module will
generate one.

Since an accurate Newsgroups header is needed to correctly munge the IDs,
this module should run after any modules responsible for generating that.
It also needs the final Message-ID and References header, so it should run
after the mailtonews module since that module may promote message IDs from
In-Reply-To into References and discard existing Message-ID headers.

This module takes one optional configuration directive:

=over 4

=item mungeids REGEX [REGEX ...]

C<REGEX> is a regex matching newsgroups which should be used to make up
the prefix that we'll be adding to message IDs.  If this configuration
directive is supplied, then only newsgroups matching one of the supplied
regexes will be used to construct the prefix.  By default, all newsgroups
in the Newsgroups header will be used in the prefix.

=back

mungeids takes no arguments and cannot fail.

=head2 newsgroups

Construct the Newsgroups header for a message.  This is the general module
for this purpose, and is considerably more flexible than the simple
handling built into the mailtonews module.

The guiding purpose of this module is to attempt to deal with crossposting
correctly in the context of a generic mail to news gateway (where incoming
mail messages may not, unlike submissions for moderated groups, already
have Newsgroups headers in them).  We're ambitious, though, and try to
also handle the case where we do have an incoming Newsgroups header.

All of the gory details of what this module does in every circumstance are
documented in the comment block at the beginning of the code.  If you're
really curious, I'd recommend reading that.  It's far more information
than most people are going to want to know.  Here's the briefer summary:

The program tells this module what the primary newsgroup is.  This should
be the group to which we should post by default.  In other words, suppose
you have several different mail to news gateway addresses for several
different groups which all feed into the same program.  To use this
module, configure such a system so that the program knows what address a
given message arrived via (either by having access to the envelope
recipient through using qmail, using procmail as a delivery agent, or some
other method, or by having each mail to news gateway alias give the
program the name of its primary newsgroup on the command line).  The
program tells this module what group that is by providing it as an
argument to the module.

We also have a list of associations between addresses and newsgroups given
to us in configuration directives.  An address can either be a literal
address (which will be matched exactly, albeit without regard to case) or
a regex.  Note that although this module wants to know about as many of
those associations as possible, it can deal with not knowing about them
all and will generally do the right thing.  It will just tend to multipost
when it could have crossposted unless it knows which addresses correspond
to which newsgroups.

If the incoming message doesn't have a Newsgroups header, this module will
construct one by inspecting the To and Cc headers and seeing where the
message was sent.  If the primary group corresponds to one of the
addresses in the To and Cc headers, then it will crosspost between the
groups corresponding to the recognized addresses in the To and Cc headers
(with the caveat given below).  Otherwise, it assumes that the address via
which we received the message either is one we don't know about or was
Bcc'd and posts only to the primary group.

There is one problem with this.  If a message is addressed to a number of
different addresses, all of which eventually gate to a newsgroup, if all
of those gateways do this analysis and crosspost, the article ends up
multiposted (one copy for each group crossposted to).  The solution to
this problem is for one of the instances of the gateway program decide to
post and all of the other ones exit quietly.  We support this by returning
the error "Not primary instance" if the primary group is in the list
formed from the To and Cc headers but isn't the first group in that list.
We assume that we'll also get the mail going to the address corresponding
to the first group in that list, and that instance will do the
crossposting.  A program using this module should probably exit silently
if we return "Not primary instance".

If the message does contain a Newsgroups header, then what we do depends
on whether our primary group is among the groups in that header, and
whether any of the addresses in the To or Cc headers correspond to groups
in that header.  The exact rules are very complicated (see the source
comments), but we ignore the Newsgroups header entirely if neither the
primary group nor the groups corresponding to addresses in the To and Cc
headers occur in it, honor it if the primary group is in it, and post only
to the groups corresponding to addreses in the To and Cc headers that
aren't in the Newsgroups header if some addresses in the To and Cc headers
correspond to groups in the Newsgroups header but our primary group isn't
in that header.  We also return "Not primary instance" where appropriate
to ensure that a post to a given set of newsgroups is only done once.

(Yes, this is a simplification.  See the source comments for the B<real>
complexity.)

We also rename Message-ID to X-Original-Message-ID if we have some
expectation that otherwise we'll clash with another post (either made by
another instance of the same program or posted elsewhere).  We also rename
Newsgroups to X-Original-Newsgroups if we don't honor the supplied header.
Note that this module goes to some length to avoid renaming Message-ID
unless necessary; if you want to drop all incoming message IDs and
generate new ones, you should do that using the headers module.

We take an argument specifying the primary group, and we take one
configuration directive in one of three forms:

=over 4

=item group NEWSGROUP [ ADDRESS | /PATTERN/ ]

Adds C<NEWSGROUP> to the list of valid newsgroups and optionally
associates it with either C<ADDRESS> or C<PATTERN>.  C<ADDRESS> is a
literal string that (case-insensitively) exactly matches the address
associated with C<NEWSGROUP>.  C<PATTERN> is a Perl regex that matches
addresses associated with C<NEWSGROUP>.

=item group /PATTERN/

Tells this module to consider any newsgroup matching C<PATTERN> to be
valid to crosspost to.  Note that this directive doesn't set up any
address to group mappings, just changes what groups are allowed in a
pre-existing Newsgroups header.

=item group FILE /PATTERN/

Tells this module to add all newsgroups matching C<PATTERN> in the file
C<FILE> to the list of valid newsgroups for crossposts.  Note that this
directive doesn't set up any address to group mappings, just changes what
groups are allowed in a pre-existing Newsgroups header.  C<FILE> must
be an absolute path (i.e., it must begin with /).

=back

In all of the above, the /s around C<PATTERN> arguments are required, as
they allow unambiguous parsing of the configuration file directives.

There are three possible failure messages returned by this module:

=over 4

=item Invalid crossposted group %s

The incoming message already had a Newsgroups header which included a
group that wasn't on the list of allowable newsgroups for crossposting.

=item Not primary instance

Although the primary group is among the groups we would normally post to,
it isn't the "first" such group.  This probably means that multiple copies
of the message were received by different gateways and this instance
should exit silently since another instance will be doing the posting.

=back

In addition to that, there are two possible fatal errors that can occur
during the parsing of the configuration file and a third at the time of
parsing the incoming message.  These errors are passed to the
News::Gateway error() method.

=over 4

=item Invalid regex /%s/: %s

An error occurred while compiling the given regex from a C<PATTERN>
argument to a configuration directive.

=item Can't open group file %s: %s

An error occurred while attempting to open a file from a C<FILE> argument
to a configuration directive.

=item newsgroups module missing required argument

The newsgroups module requires an argument giving the primary newsgroup,
or all of the tricks we use to figure out crossposting and where to post
won't work correctly.

=back

=head2 newstomail

Rewrites a news article into a mail message, using a set of newsgroup to
e-mail address mappings to determine which addresses to mail the resulting
message to.

This module does the following:  Checks to make sure there's a Newsgroups
header; drops Bcc and Resent-Bcc headers in the message; renames To, Cc,
Apparently-To, Resent-To, Resent-Cc, Return-Path, and Sender headers (if
present) to the same name prefixed by X-Original- since they may be
misinterpreted by e-mail software, and sets a new To header consisting of
the e-mail addresses corresponding to all the groups in the Newsgroups
header that have defined mappings.  Duplicate addresses are stripped out.

This module takes one configuration file directive:

=over 4

=item newstomail FILENAME

Specifies the file from which the list of newsgroup to address mappings
should be read, in the form newsgroup, whitespace, address.  If the
filename ends in .db, then the file will be assumed to be a Berkeley
database instead, with newsgroups as the keys and addresses as the values.

=back

Two failure messages are possible:

=over 4

=item Missing required Newsgroups header

The incoming message doesn't have a Newsgroups header.  This module only
handles posts and needs the Newsgroups header to determine what addresses
to which to send the message.

=item No newsgroup with a mapping

None of the newsgroups in the Newsgroups header had a mapping to an e-mail
address, so there's nowhere to send the message.

=back

This module takes no arguments.

=head2 nobinaries

Checks the message to see if it is or contains a binary and rejects it if
it is.  The following checks are performed:  Ensure that no Content-Type
header in the headers or the body contains the strings "application",
"image", "audio", or "video"; ensure that no Content-Transfer-Encoding
header in the headers or the body is equal to "base64", and ensure that
encoded lines do not exceed 50% of the number of lines in the body for any
article with at least 40 lines.

An encoded line is defined as a line beginning with an M (with optional
leading whitespace or quoting characters) and exactly 60 or 61 characters
in length, or a line containing no spaces, not starting with M or ~, and
between 59 and 80 characters in length.

Lines meeting those criteria are counted separately (the first as
potential uuencoding, the second as potential base64 encoding), and if the
count of lines in any one category is over half of the total body lines,
the message is rejected.

The rejection messages are:

=over 4

=item base64 encoded

The message contains a Content-Transfer-Encoding header with a value of
"base64".

=item Invalid content type

The message contains a Content-Type header that contains one of the
strings "application", "image", "audio", or "video".

=item Apparently uuencoded

The message body is at least 40 lines and more than 50% of those lines
start with M (modulo whitespace and quoting) and are exactly 60 or 61
characters long.

=item Apparently base64-encoded

The message body is at least 40 lines and more than 50% of those lines
contain no spaces, do not begin with M or ~, and are between 59 and 80
characters long.

=back

This article takes no arguments or configuration directives.

=head2 previoushop

Adds the previous host through which a mail message passed to the Path
header of the message.  It does this by looking through the raw Received
headers, finding the first Received header that specifies a host from
which the message was received, and prepending it to the Path header (or
creating a new Path header if none already exists).

If no Received headers that contain a "from" clause are found (which may
be the case for mail originating locally), nothing is done.

The reason for this module is to support news to mail and mail to news
gateways for the same unmoderated newsgroup.  If the mail to news address
is subscribed to a mailing list and posts to the newsgroup are sent back
to the mailing list, a loop will be created unless something is done to
stop it.  One method of stopping such loops is to alias the hostname of
the mailing list machine out of the feed to the news to mail gateway in
the news server and then use this module to put the hostname of the
mailing list machine in the Path of all articles posted through the mail
to news gateway.

This module takes no arguments or configuration directives and cannot
fail.

=head2 whitelist

Checks to make sure that the incoming message is from a poster in a file
of valid posters.  The most common use of this module is in conjunction
with human moderation; all messages from pre-approved posters are sent
straight through via the robomoderator and the rest are relayed to a human
moderator for hand checking.

This module takes one configuration directive:

=over 4

=item whitelist FILE

C<FILE> is a list of poster addresses, one per line.  Only the address
should be included in this file, not the name or other comments.  Files
ending in C<.db> are reserved for future implementations (eventually,
these will automatically be recognized as Berkeley db files for handling
large lists of posters).

=back

There is one possible failure message:

=over 4

=item Unknown poster %s

The message is from a poster address which isn't in the whitelist of known
addresses.

=back

In addition, during parsing of the configuration directives, one fatal
error, which would be passed to error(), is possible:

=over 4

=item Can't open whitelist file %s: %s

An error occurred while trying to open a file given as a C<FILE> argument
to a configuration directive.

=back

This module takes no arguments.

=head1 UTILITY MODULES

The following modules are also part of the standard distribution.  Rather
than rewrite or check messages, they provide additional methods that can
be used by programs, other modules, even or the News::Gateway core code.
Methods supplied by utility modules can be called like any other
News::Gateway method.

=head2 config

This module contains the code for reading and parsing configuration
directives and calling the appropriate module callback hooks.  Nearly all
News::Gateway programs will make use of this module.

B<WARNING>: Information in configuration directives is B<trusted> by
modules, and modules may go so far as to eval things from configuration
directives as Perl code.  This means that anyone who has access to the
configuration directives read by this module has as much control over what
a program does as someone who can edit the source.  Any necessary sanity
or security checking on the content of configuration directives must be
done B<before> they are passed off to modules, and C<config_file()> does
no such checking.

=over 4

=item config (DIRECTIVE, ARGUMENT [, ARGUMENT ...])

Calls the registered module callbacks for all modules who have expressed
an interest in C<DIRECTIVE>.  This bypasses any parsing, and the arguments
are given to the callbacks exactly as passed in.  C<DIRECTIVE> is
case-insensitive.

=item config_file (FILE | HANDLE)

Reads in configuration directives from the supplied file name or file
handle (which can be either an object reference or a reference to a
typeglob) until end of file is reached.

Blank lines and lines beginning with a C<#> are ignored.  All other lines
are parsed into a directive (the first whitespace-separated word) and some
number of arguments, and then the parse results are passed along to the
module which has registered interest in that directive.  If an argument
contains embedded whitespace, it can be enclosed in double quotes.  A
backslash will escape the next character, whatever it is.  A line is
considered to be continued on the next line if it ends in a backslash, and
if the line continuation occurs inside double quotes, a literal newline
will be part of the string.

For example, the following configuration directive tells the C<headers>
module to add a new header named X-Comment with the content "This is a
long comment about this newsgroup":

    header x-comment add \
           "This is a long comment about this newsgroup"

Multiple configuration files can be read by calling C<config_file()>
multiple times.  One handy trick for simple scripts using News::Gateway is
to put the configuration lines at the end of the script after an __END__
and then pass them to News::Gateway with:

    $gateway->config_file (\*DATA);

C<config_file()> should be called before any C<apply()> it is supposed to
affect.

=item config_line (LINE)

Parses and handles a single configuration line, just as if it were read
from a file (in fact, C<config_file()> calls this method for each
non-blank, non-comment line).  This can be used to feed individual
configuration lines to the News::Gateway object without having to give it
a file or file handle.

=item config_parse (LINE)

Parses a configuration file line, returning the results as an array.  This
is the method used by C<config_file()> and C<config_line()> to parse
configuration files.  This probably isn't generally useful.

=back

These methods adds two additional fatal error messages, which may be
passed to C<error()>.

=over 4

=item Parse error in %s

A configuration line was unable to be parsed.  The most likely cause is
unbalanced double quotes.

=item Unknown configuration directive %s

News::Gateway encountered a configuration directive that no module had
expressed interest in.  Chances are you made a typo in your configuration
file or forgot to register a module with C<modules()> that you were
planning on using.

=back

=head2 mail

This module separates out various functions related to sending mail into
their own module so that it can be autoloaded on demand rather than always
compiled.  The following methods are included:

=over 4

=item mail ([ ADDRESS [, ADDRESS ...]])

The message in its current form will be mailed to C<ADDRESS>, and the
method will return true on success, false on failure.  If C<ADDRESS> is
not supplied, the message is mailed to the addresses in the standard mail
headers (To, Cc, Bcc, and the Resent-* varients depending on your MTA), so
this method should also be used after news to mail rewrites have taken
place.

B<NOTE: The old mail_forward() method is DEPRECATED.>

=item mail_bounce (ERROR)

Bounces the message by printing C<ERROR> to stderr and then exiting with
the permanent failure status code (as defined in the beginning of the
News::Gateway source).  This should only be used as a last resort,
particularly for robomoderators, since the bounces will go to the envelope
sender address which is often not the same as the poster.

=item mail_error (ERROR)

Attempts to send an error message to the sender of the original message,
cc'd to the maintainer address.  Replies will be set to the maintainer
address, the subject will be "failure notice," and a Precedence header of
"junk" will be added, which can be used to avoid loops.

If no From or Reply-To header has been seen, or if sending the mail fails,
this method will call mail_bounce() with the same error message instead.

The message sent will be:

    Hi.  I'm afraid that I was unable to post your message.  This
    is a fatal error; I've given up.  A copy of this report is
    being sent to my maintainer.  The error message I received was:

    ERROR

    --- Below this line is a copy of the message.

followed by the original article headers and the body, with ERROR replaced
by C<ERROR>.  Note that if C<ERROR> should be word-wrapped in the message,
this must be done by the caller.

=item mail_filereply (FILENAME [, SOURCES])

Mails a News::FormReply-style message to the author of the incoming
message.  See L<News::FormReply> for more information.  C<FILENAME> is the
path to the file to use to construct the reply, and C<SOURCES> for
News::FormReply variables can optionally be provided.  The following
variables are exposed to the form message by default:

    @BODY        Message body, possibly munged by previous modules.
    @HEADERS     Current message headers.
    @OLDHEADERS  Original message headers.
    $SUBJECT     Original subject line.
    $MAINTAINER  Maintainer of this gateway.

Returns true if sending the mail succeeded, false if it failed.

=back

=head2 post

Implements all methods related to posting articles.  The following methods
are included:

=over 4

=item post ([ SERVER ])

Posts the article using the NNTP POST command.  Under most circumstances,
this is how you want to post an article.  SERVER, if specified, may be
either the name or IP address of a news server to use for posting or a
reference to an open Net::NNTP connection to be used.  undef is returned
on success; the error message is returned on failure.

Before this method is called, the message headers should be rewritten into
ones appropriate for a news message; this is one of the purposes of the
modules.

=item post_ihave ([ SERVER ])

Posts the article using the NNTP IHAVE command.  This connects to a news
server as a transfer server rather than as a client; under most
circumstances this is not what you want.  Unless you already know what
IHAVE is, you may not want to use this.  In order to use this posting
method, the article already has to have all necessary headers to be a
properly formatted news article, including Path and Message-ID; unlike
POST, IHAVE will not automatically fill in missing headers.

As above, SERVER if specified may be either the name or IP address of a
news server to use for posting or a reference to an open Net::NNTP
connection.  undef is returned on success; the error message is returned
on failure.

=item post_program (COMMAND [, ARGUMENT ... ])

Posts the article by running a program and passing the article to it on
stdin.  This can be used to post via inews or rnews if you so wish,
although using the post() method is recommended under most circumstances.
The arguments to the command should be a list of strings; this command
does not fork a shell so shell metacharacters will not be interpreted as
such.

If the program exits with status 0, success is assumed and this method
returns undef.  If the program exits with non-zero status or some other
error occurs, this method returns the output of the program (both stdout
and stderr) and an error message.  The output and error message will be
newline-terminated.

=back

=head1 DIAGNOSTICS

In addition to the failure messages generated by modules (listed above),
the following messages may be generated by the News::Gateway core code and
would be passed to the C<error()> method.

=over 4

=item Autoload of %s failed: %s

A method was called that doesn't exist in the core code and was unable to
be loaded from a module.  Chances are you made a typo when referring to a
News::Gateway method.

=item Cannot open file %s: %s

News::Gateway attempted to open a file (probably the configuration file)
and failed for the given reason.

=item Unable to load %s: %s

One of the modules (either rewrite or utility) you used requires an
external Perl module and was unable to load it.

=item Unknown module %s

You told C<modules()> to load a module that has no hooks registered.
News::Gateway has to know about every module that it can use; you can't
just copy in a new module with the others and have it work.  Instead, put
the module in the F<modules> directory before the build process.

=back

=head1 BUGS

Many.  This is an B<alpha> release, with all that's implied by that in
terms of bugs, interface instability, and other such concerns.  Use at
your own risk, but if you encounter a bug please let me know.  Patches in
particular are gratefully accepted.

=head1 NOTES

If you are interested in being notified of new releases, helping with
development, developing, testing, or using new modules, or even just
getting tips on how to use this module, there is a mailing list available.
To subscribe, send mail to majordomo@eyrie.org with:

    subscribe gateway-users

in the body.

New patches and modules are always welcome, whether from people on the
list or not, and can be mailed to the address listed below.

=head1 SEE ALSO

L<News::Article>

=head1 AUTHOR

Russ Allbery E<lt>rra@stanford.eduE<gt>.

=head1 HISTORY

News::Gateway started as a simple mail to news Perl script, written right
after the cs.utexas.edu mail to news gateway closed and just a quick hack
for the use of people who had previously used that gateway to post to
rec.arts.comics.creative and alt.comics.lnh.  Over time, it became a
general mail to news gateway with a fairly broad set of features.

Eventually, it became obvious that it really B<should> have been a module
rather than a script, and it also became clear that it could be used as
the core of a robomoderator.  Since there was no real general-purpose
robomoderation software around that was nearly as flexible as I thought a
package B<should> be, I decided to be ambitious and try to solve the
general problem.

Then I took a complete detour to rewrite the entire thing again to use
Andrew Gierth's News::Article class, which is what I should have been
doing from the beginning.  And now, finally, it's getting closer to what
I'd envisioned.
