1MAKEHISTORY(8) InterNetNews Documentation MAKEHISTORY(8)
2
6 makehistory - Initialize or rebuild INN history database
7
9 makehistory [-abFIOSx] [-f filename] [-l count] [-L load-average] [-s
10 size] [-T tmpdir]
11
13 makehistory rebuilds the history(5) text file, which contains a list of
14 message-IDs of articles already seen by the server. It can also be
15 used to rebuild the overview database. Note that even though the dbz
16 indices for the history file are also rebuilt by makehistory, it is
17 useful to run makedbz(8) after makehistory(8) in order to improve the
18 efficiency of the indices (makehistory does not know how large to make
19 the hash table at first run, unless the size is given by the -s flag).
20 The default location of the history text file is pathdb/history; to
22 specify an alternate location, use the -f flag.
23 By default, makehistory will scan the entire spool, using the storage
25 manager, and write a history line for every article. To also generate
26 overview information, use the -O flag.
27 WARNING: If you're trying to rebuild the overview database, be sure to
29 stop innd(8) and delete or zero out the existing database before you
30 start for the best results. An overview rebuild should not be done
31 while the server is running. Unless the existing overview is deleted,
32 you may end up with problems like out-of-order overview entries,
33 excessively large overview buffers, and the like.
34 If ovmethod in inn.conf is "ovdb", you must have the ovdb processes
36 running while rebuilding overview. ovdb needs them available while
37 writing overview entries. You can start them by hand separate from the
38 rest of the server by running ovdb_init; see ovdb_init(8) for more
39 details.
40
42 -a Append to the history file rather than generating a new one. If
43 you append to the main history file, make sure innd(8) is throttled
44 or not running, or you can corrupt the history.
45 -b Delete any messages found in the spool that do not have valid
47 Message-ID: headers in them.
48 -F Fork a separate process to flush overview data to disk rather than
50 doing it directly. The advantage of this is that it allows
51 makehistory to continue to collect more data from the spool while
52 the first batch of data is being written to the overview database.
53 The disadvantage is that up to twice as much temporary disk space
54 will be used for the generated overview data. This option only
55 makes sense in combination with -O. With buffindexed, the overchan
56 program is invoked to write overview.
57 -f filename
59 Rather than writing directly to pathdb/history, instead write to
60 filename, also in pathdb.
61 -I Don't store overview data for articles numbered lower than the
63 lowest article number in active. This is useful if there are for
64 whatever reason old articles on disk that shouldn't be available to
65 readers or put into the overview database.
66 -l count
68 This option specifies how many articles to process before writing
69 the accumulated overview information out to the overview database.
70 The default is 10000. Since overview write performance is faster
71 with sorted data, each "batch" gets sorted. Increasing the batch
72 size with this option may further improve write performance, at the
73 cost of longer sort times. Also, temporary space will be needed to
74 store the overview batches. At a rough estimate, about 300 * count
75 bytes of temporary space will be required (not counting temp files
76 created by sort(1)). See the description of the -T option for how
77 to specify the temporary storage location. This option has no
78 effect with buffindexed, because buffindexed does not need sorted
79 overview and no batching is done.
80 -L load-average
82 Temporarily pause activities if the system load average exceeds the
83 specified level load-average. This allows makehistory to run on a
84 system being used for other purposes without monopolizing system
85 resources and thus making the response time for other applications
86 unacceptably slow. Using nice(1) does not help much for that
87 because the problem comes from disk I/O usage, and ionice(1) is not
88 always available or efficient.
89 -O Create the overview database as well as the history file. Overview
91 information is only required if the server supports readers; it is
92 not needed for a transit-only server (see enableoverview in
93 inn.conf(5)). If you are using the buffindexed overview storage
94 method, erase all of your overview buffers before running
95 makehistory with -O.
96 -S Rather than storing the overview data into the overview database,
98 just write it to standard output in a form suitable for feeding to
99 overchan later if wished. When this option is used, -F, -I, -l,
100 and -T are ignored. This option only makes sense in combination
101 with -O.
102 -s size
104 Size the history database for approximately size pairs. Accurately
105 specifying the size is an optimization that will create a more
106 efficient database. (The size should be the estimated eventual
107 size of the history file, typically the size of the old file, in
108 lines.)
109 -T tmpdir
111 If -O is given, makehistory needs a location to write temporary
112 overview data. By default, it uses pathtmp, set in inn.conf, but
113 if this option is given, the provided tmpdir is used instead. This
114 is also used for temporary files created by sort(1) (which is
115 invoked in the process of writing overview information since sorted
116 overview information writes faster). By default, sort usually uses
117 your system temporary directory; see the sort(1) man page on your
118 system to be sure.
119 -x If this option is given, makehistory won't write out history file
121 entries. This is useful mostly for building overview without
122 generating a new history file.
123
125 Here's a typical example of rebuilding the entire history and overview
126 database, removing broken articles in the news spool. This uses the
127 default temporary file locations and should be done while innd isn't
128 running (or is throttled).
129 makehistory -b -f history.n -O -l 30000 -I
131 This will rebuild the overview (if using buffindexed, erase the
133 existing overview buffers before running this command) and leave a new
134 history file as "history.n" in pathdb. To preserve all of the history
135 entries from the old history file that correspond to rejected articles
136 or expired articles, follow the above command with:
137 cd <pathdb>
139 awk 'NF == 2 { print }' < history >> history.n
140 (replacing the path with your pathdb, if it isn't the default). Then
142 look over the new history file for problems and run:
143 makedbz -s `wc -l < history.n` -f history.n
145 Then rename all of the files matching "history.n.*" to "history.*",
147 replacing the current history database and indices. After that, it's
148 safe to unthrottle innd.
149 For a simpler example:
151 makehistory -b -f history.n -I -O
153 will scan the spool, removing broken articles and generating history
155 and overview entries for articles missing from history.
156 To just rebuild overview:
158 makehistory -O -x -F
160
162 pathdb/history
163 This is the default output file for makehistory.
164 pathtmp
166 Where temporary files are written unless -T is given.
167
169 Originally written by Rich $alz <rsalz@uunet.uu.net> for InterNetNews
170 and updated by various other people since.
171
173 active(5), ctlinnd(8), history(5), inn.conf(5), innd(8), libinn_dbz(3),
174 makedbz(8), ovdb_init(8), overchan(8).
175INN 2.6.5 2022-02-18 MAKEHISTORY(8)