1PUBLIC-INBOX-EXTINDEX(1)   public-inbox user manual   PUBLIC-INBOX-EXTINDEX(1)
2
3
4

NAME

6       public-inbox-extindex - create and update external search indices
7

SYNOPSIS

9       public-inbox-extindex [OPTIONS] EXTINDEX_DIR INBOX_DIR...
10
11       public-inbox-extindex [OPTIONS] [EXTINDEX_DIR] --all
12

DESCRIPTION

14       public-inbox-extindex creates and updates an external search and
15       overview database used by the read-only public-inbox PSGI (HTTP), NNTP,
16       and IMAP interfaces.  This requires either the Search::Xapian XS
17       bindings OR the Xapian SWIG bindings, along with DBD::SQLite and DBI
18       Perl modules.
19

OPTIONS

21       -j JOBS
22       --jobs=JOBS
23       --no-fsync
24       --dangerous
25       --rethread
26       --max-size SIZE
27       --batch-size SIZE
28           These switches behave as they do for public-inbox-index(1)
29
30       --all
31           Index all "publicinbox" entries in "PI_CONFIG".
32
33           "publicinbox" entries indexed by "public-inbox-extindex" can have
34           full Xapian searching abilities with the per-"publicinbox"
35           "indexlevel" set to "basic" and their respective Xapian ("xap15" or
36           "xapian15") directories removed.  For multiple public-inboxes where
37           cross-posting is common, this allows significant space savings on
38           Xapian indices.
39
40       --gc
41           Perform garbage collection instead of indexing.  Use this if
42           inboxes are removed from the extindex, or if messages are purged or
43           removed from some inboxes.
44
45       --reindex
46           Forces a re-index of all messages in the extindex.  This can be
47           used for in-place upgrades and bugfixes while read-only server
48           processes are utilizing the index.  Keep in mind this roughly
49           doubles the size of the already-large Xapian database.
50
51           The extindex locks will be released roughly every 10s to allow
52           public-inbox-mda(1) and public-inbox-watch(1) processes to write to
53           the extindex.
54
55       --fast
56           Used with "--reindex", it will only look for new and stale entries
57           and not touch already-indexed messages.
58

FILES

60       public-inbox-extindex-format(5)
61

CONFIGURATION

63       public-inbox-extindex does not currently write to the
64       public-inbox-config(5) file, configuration may be entered manually.
65       The extindex name of "all" is a special case which corresponds to
66       indexing "--all" inboxes.  An example for "--all" is as follows:
67
68               [extindex "all"]
69                       topdir = /path/to/extindex_dir
70                       url = all
71                       coderepo = foo
72                       coderepo = bar
73
74       See public-inbox-config(5) for more details.
75

ENVIRONMENT

77       PI_CONFIG
78               Used to override the default "~/.public-inbox/config" value.
79
80       XAPIAN_FLUSH_THRESHOLD
81               The number of documents to update before committing changes to
82               disk.  This environment is handled directly by Xapian, refer to
83               Xapian API documentation for more details.
84
85               Setting "XAPIAN_FLUSH_THRESHOLD" or
86               "publicinbox.indexBatchSize" for a large "--reindex" may cause
87               public-inbox-mda(1), public-inbox-learn(1) and
88               public-inbox-watch(1) tasks to wait long and unpredictable
89               periods of time during "--reindex".
90
91               Default: none, uses "publicinbox.indexBatchSize"
92

UPGRADING

94       Occasionally, public-inbox will update it's schema version and require
95       a full index by running this command.
96

CONTACT

98       Feedback welcome via plain-text mail to <mailto:meta@public-inbox.org>
99
100       The mail archives are hosted at <https://public-inbox.org/meta/> and
101       <http://4uok3hntl7oi7b4uf4rtfwefqeexfzil2w6kgk2jn5z2f764irre7byd.onion/meta/>
102
104       Copyright all contributors <mailto:meta@public-inbox.org>
105
106       License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
107

SEE ALSO

109       Search::Xapian, DBD::SQLite
110
111
112
113public-inbox.git                  1993-10-02          PUBLIC-INBOX-EXTINDEX(1)
Impressum