=head1 NAME

makehistory - Initialize or rebuild INN history database

=head1 SYNOPSIS

B<makehistory> [B<-abeFIOx>] [B<-f> I<filename>] [B<-l> I<count>]
[B<-T> I<tmpdir>] [B<-s> I<size>]

=head1 DESCRIPTION

B<makehistory> rebuilds the history(5) text file, which contains a list of
Message-IDs of articles already seen by the server.  It can also be used
to rebuild the overview database.  Note that the dbz(3) indexes for the
history file are rebuilt by makedbz(8), not by B<makehistory> as in
earlier versions of INN.

The default location of the history text file is I<pathdb>/history; to
specify an alternate location, use the B<-f> flag.

By default, B<makehistory> will scan the entire spool, using the storage
manager, and write a history line for every article.  To also generate
overview information, use the B<-O> flag.

WARNING: If you're trying to rebuild the overview database, be sure to
stop innd(8) and delete or zero out the existing database before you start
for the best results.  An overview rebuild should not be done while the
server is running.  Unless the existing overview is deleted, you may end
up with problems like out-of-order overview entries, excessively large
overview buffers, and the like.

If I<ovmethod> in F<inn.conf> is C<ovdb>, you must have the ovdb processes
running while rebuilding overview.  ovdb needs them available while
writing overview entries.  You can start them by hand separate from the
rest of the server by running B<ovdb_init>; see ovdb_init(8) for more
details.

=head1 OPTIONS

=over 4

=item B<-a>

Append to the history file rather than generating a new one.  If you
append to the main history file, make sure innd(8) is throttled or not
running, or you can corrupt the history.

=item B<-b>

Delete any messages found in the spool that do not have valid Message-ID
headers in them.

=item B<-e>

Compute Bytes headers which is used for overview data.  This option is valid
only if B<-O> flag is specified and F<overview.fmt> includes C<Bytes:>.

=item B<-f> I<filename>

Rather than writing directly to I<pathdb>/history, instead write to
I<filename>.

=item B<-F>

Fork a separate process to flush overview data to disk rather than doing
it directly.  The advantage of this is that it allows B<makehistory> to
continue to collect more data from the spool while the first batch of data
is being written to the overview database.  The disadvantage is that up to
twice as much temporary disk space will be used for the generated overview
data.  This option only makes sense in combination with B<-O>.  With
C<buffindexed>, the C<overchan> program is invoked to write overview.

=item B<-I>

Don't store overview data for articles numbered lower than the lowest
article number in F<active>.  This is useful if there are for whatever
reason old articles on disk that shouldn't be available to readers or put
into the overview database.

=item B<-l> I<count>

This option specifies how many articles to process before writing the
accumulated overview information out to the overview database.  The
default is C<10000>.  Since overview write performance is faster with
sorted data, each "batch" gets sorted.  Increasing the batch size
with this option may further improve write performance, at the cost
of longer sort times.  Also, temporary space will be needed to store
the overview batches.  At a rough estimate, about 300 * I<count> bytes
of temporary space will be required (not counting temp files created
by sort(1)).  See the description of the B<-T> option for how to
specify the temporary storage location.  This option has no effect
with C<buffindexed>, because C<buffindexed> does not need sorted
overview and no batching is done.

=item B<-s> I<size>

Size the history database for approximately I<size> pairs.  Accurately
specifying the size is an optimization that will create a more
efficient database.  (The size should be the estimated eventual size
of the F<history> file, typically the size of the old file, in lines.)

=item B<-O>

Create the overview database as well as the history file.  Overview
information is only required if the server supports readers; it is not
needed for a transit-only server (see I<enableoverview> in inn.conf(5)).
If you are using the C<buffindexed> overview storage method, erase all of
your overview buffers before running B<makehistory> with B<-O>.

=item B<-T> I<tmpdir>

If B<-O> is given, B<makehistory> needs a location to write temporary
overview data.  By default, it uses I<pathtmp>, set in F<inn.conf>, but if
this option is given, the provided I<tmpdir> is used instead.  This is
also used for temporary files created by sort(1) (which is invoked in the
process of writing overview information since sorted overview information
writes faster).  By default, sort usually uses your system temporary
directory; see the sort(1) man page on your system to be sure.

=item B<-x>

If this option is given, B<makehistory> won't write out history file
entries.  This is useful mostly for building overview without generating
a new history file.

=back

=head1 EXAMPLES

Here's a typical example of rebuilding the entire history and overview
database, removing broken articles in the news spool.  This uses the
default temporary file locations and should be done while innd isn't
running (or is throttled).

    makehistory -b -f history.n -O -l 30000 -I

This will rebuild the overview (if using C<buffindexed>, erase the
existing overview buffers before running this command) and leave a new
history file as C<history.n> in I<pathdb>.  To preserve all of the history
entries from the old history file that correspond to rejected articles or
expired articles, follow the above command with:

    cd /usr/local/news/db
    awk 'NF == 2 { print }' < history >> history.n

(replacing the path with your I<pathdb>, if it isn't the default).  Then
look over the new history file for problems and run:

    makedbz -s `wc -l < history` -f history.n

Then rename all of the files matching C<history.n.*> to C<history.*>,
replacing the current history database and indexes.  After that, it's safe
to unthrottle innd.

For a simpler example:

    makehistory -b -f history.n -I -O

will scan the spool, removing broken articles and generating history and
overview entries for articles missing from history.

To just rebuild overview:

    makehistory -O -x -F

=head1 FILES

=over 4

=item inn.conf

Read for I<pathdb>, I<pathtmp>, and other settings.

=item I<pathdb>/history

This is the default output file for B<makehistory>.

=item I<pathtmp>

Where temporary files are written unless B<-T> is given.

=back

=head1 HISTORY

Originally written by Rich $alz <rsalz@uunet.uu.net> for InterNetNews and
updated by various other people since.

$Id: makehistory.pod 6400 2003-07-12 19:26:58Z rra $

=head1 SEE ALSO

dbz(3), active(5), history(5), inn.conf(5), ctlinnd(8), innd(8),
makedbz(8), ovdb_init(8), overview.fmt(5).

=cut