1PUBLIC-INBOX-EXTINDEX(1) public-inbox user manual PUBLIC-INBOX-EXTINDEX(1)
2
3
4
6 public-inbox-extindex - create and update external search indices
7
9 public-inbox-extindex [OPTIONS] EXTINDEX_DIR INBOX_DIR...
10
11 public-inbox-extindex [OPTIONS] [EXTINDEX_DIR] --all
12
14 public-inbox-extindex creates and updates an external search and
15 overview database used by the read-only public-inbox PSGI (HTTP), NNTP,
16 and IMAP interfaces. This requires either the Search::Xapian XS
17 bindings OR the Xapian SWIG bindings, along with DBD::SQLite and DBI
18 Perl modules.
19
21 -j JOBS
22 --jobs=JOBS
23 ... TODO, see public-inbox-index(5)
24
25 --all
26 Index all "publicinbox" entries in "PI_CONFIG".
27
28 "publicinbox" entries indexed by "public-inbox-extindex" can have
29 full Xapian searching abilities with the per-"publicinbox"
30 "indexlevel" set to "basic" and their respective Xapian ("xap15" or
31 "xapian15") directories removed. For multiple public-inboxes where
32 cross-posting is common, this allows significant space savings on
33 Xapian indices.
34
35 --gc
36 Perform garbage collection instead of indexing. Use this if
37 inboxes are removed from the extindex, or if messages are purged or
38 removed from some inboxes.
39
40 --reindex
41 Forces a re-index of all messages in the extindex. This can be
42 used for in-place upgrades and bugfixes while read-only server
43 processes are utilizing the index. Keep in mind this roughly
44 doubles the size of the already-large Xapian database.
45
46 The extindex locks will be released roughly every 10s to allow
47 public-inbox-mda(1) and public-inbox-watch(1) processes to write to
48 the extindex.
49
50 --fast
51 Used with "--reindex", it will only look for new and stale entries
52 and not touch already-indexed messages.
53
55 public-inbox-extindex-format(5)
56
58 public-inbox-extindex does not currently write to the
59 public-inbox-config(5) file, configuration may be entered manually.
60 The extindex name of "all" is a special case which corresponds to
61 indexing "--all" inboxes. An example for "--all" is as follows:
62
63 [extindex "all"]
64 topdir = /path/to/extindex_dir
65 url = all
66 coderepo = foo
67 coderepo = bar
68
69 See public-inbox-config(5) for more details.
70
72 PI_CONFIG
73 Used to override the default "~/.public-inbox/config" value.
74
75 XAPIAN_FLUSH_THRESHOLD
76 The number of documents to update before committing changes to
77 disk. This environment is handled directly by Xapian, refer to
78 Xapian API documentation for more details.
79
80 Setting "XAPIAN_FLUSH_THRESHOLD" or
81 "publicinbox.indexBatchSize" for a large "--reindex" may cause
82 public-inbox-mda(1), public-inbox-learn(1) and
83 public-inbox-watch(1) tasks to wait long and unpredictable
84 periods of time during "--reindex".
85
86 Default: none, uses "publicinbox.indexBatchSize"
87
89 Occasionally, public-inbox will update it's schema version and require
90 a full index by running this command.
91
93 Feedback welcome via plain-text mail to <mailto:meta@public-inbox.org>
94
95 The mail archives are hosted at <https://public-inbox.org/meta/> and
96 <http://4uok3hntl7oi7b4uf4rtfwefqeexfzil2w6kgk2jn5z2f764irre7byd.onion/meta/>
97
99 Copyright 2021 all contributors <mailto:meta@public-inbox.org>
100
101 License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
102
104 Search::Xapian, DBD::SQLite
105
106
107
108public-inbox.git 1993-10-02 PUBLIC-INBOX-EXTINDEX(1)