1PUBLIC-INBOX-EXTINDEX(1) public-inbox user manual PUBLIC-INBOX-EXTINDEX(1)
2
3
4
6 public-inbox-extindex - create and update external search indices
7
9 public-inbox-extindex [OPTIONS] EXTINDEX_DIR INBOX_DIR...
10
11 public-inbox-extindex [OPTIONS] [EXTINDEX_DIR] --all
12
14 public-inbox-extindex creates and updates an external search and
15 overview database used by the read-only public-inbox PSGI (HTTP), NNTP,
16 and IMAP interfaces. This requires either the Search::Xapian XS
17 bindings OR the Xapian SWIG bindings, along with DBD::SQLite and DBI
18 Perl modules.
19
21 -j JOBS
22 --jobs=JOBS
23 --no-fsync
24 --dangerous
25 --rethread
26 --max-size SIZE
27 --batch-size SIZE
28 These switches behave as they do for public-inbox-index(1)
29
30 --all
31 Index all "publicinbox" entries in "PI_CONFIG".
32
33 "publicinbox" entries indexed by "public-inbox-extindex" can have
34 full Xapian searching abilities with the per-"publicinbox"
35 "indexlevel" set to "basic" and their respective Xapian ("xap15" or
36 "xapian15") directories removed. For multiple public-inboxes where
37 cross-posting is common, this allows significant space savings on
38 Xapian indices.
39
40 --gc
41 Perform garbage collection instead of indexing. Use this if
42 inboxes are removed from the extindex, or if messages are purged or
43 removed from some inboxes.
44
45 --reindex
46 Forces a re-index of all messages in the extindex. This can be
47 used for in-place upgrades and bugfixes while read-only server
48 processes are utilizing the index. Keep in mind this roughly
49 doubles the size of the already-large Xapian database.
50
51 The extindex locks will be released roughly every 10s to allow
52 public-inbox-mda(1) and public-inbox-watch(1) processes to write to
53 the extindex.
54
55 --fast
56 Used with "--reindex", it will only look for new and stale entries
57 and not touch already-indexed messages.
58
60 public-inbox-extindex-format(5)
61
63 public-inbox-extindex does not currently write to the
64 public-inbox-config(5) file, configuration may be entered manually.
65 The extindex name of "all" is a special case which corresponds to
66 indexing "--all" inboxes. An example for "--all" is as follows:
67
68 [extindex "all"]
69 topdir = /path/to/extindex_dir
70 url = all
71 coderepo = foo
72 coderepo = bar
73
74 See public-inbox-config(5) for more details.
75
77 PI_CONFIG
78 Used to override the default "~/.public-inbox/config" value.
79
80 XAPIAN_FLUSH_THRESHOLD
81 The number of documents to update before committing changes to
82 disk. This environment is handled directly by Xapian, refer to
83 Xapian API documentation for more details.
84
85 Setting "XAPIAN_FLUSH_THRESHOLD" or
86 "publicinbox.indexBatchSize" for a large "--reindex" may cause
87 public-inbox-mda(1), public-inbox-learn(1) and
88 public-inbox-watch(1) tasks to wait long and unpredictable
89 periods of time during "--reindex".
90
91 Default: none, uses "publicinbox.indexBatchSize"
92
94 Occasionally, public-inbox will update it's schema version and require
95 a full index by running this command.
96
98 Feedback welcome via plain-text mail to <mailto:meta@public-inbox.org>
99
100 The mail archives are hosted at <https://public-inbox.org/meta/> and
101 <http://4uok3hntl7oi7b4uf4rtfwefqeexfzil2w6kgk2jn5z2f764irre7byd.onion/meta/>
102
104 Copyright all contributors <mailto:meta@public-inbox.org>
105
106 License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
107
109 Search::Xapian, DBD::SQLite
110
111
112
113public-inbox.git 1993-10-02 PUBLIC-INBOX-EXTINDEX(1)