public-inbox-clone(1)

1PUBLIC-INBOX-CLONE(1)      public-inbox user manual      PUBLIC-INBOX-CLONE(1)
2
3
4

NAME

6       public-inbox-clone - "git clone --mirror" wrapper
7

SYNOPSIS

9       public-inbox-clone [OPTIONS] INBOX_URL [INBOX_DIR]
10
11       public-inbox-clone [OPTIONS] ROOT_URL [DESTINATION] # public-inbox 2.0+
12

DESCRIPTION

14       public-inbox-clone is a wrapper around "git clone --mirror" for making
15       the initial clone of a remote HTTP(S) public-inbox.  It allows cloning
16       multi-epoch v2 inboxes with a single command and zero configuration.
17
18       In public-inbox 2.0+, public-inbox-clone can create and maintain a
19       mirror of multiple inboxes or code repositories using manifest.js.gz
20       files like grok-pull(1) from grokmirror.  public-inbox-fetch(1) is NOT
21       required when using this mode.
22
23       It does not run public-inbox-init(1) nor public-inbox-index(1).  Those
24       commands must be run separately if serving/searching the mirror is
25       required.  As-is, public-inbox-clone is suitable for creating a git-
26       only backup without Xapian and SQLite indices.
27
28       When cloning a single inbox, public-inbox-clone creates a Makefile with
29       handy targets to update the inbox once indexed.  This Makefile may be
30       edited by the user; it will not be rewritten by public-inbox-fetch(1)
31       unless it is removed completely.
32
33       public-inbox-clone does not use nor require any extra configuration
34       files (not even "~/.public-inbox/config"), but it can download snippets
35       suitable for adding to any public-inbox-config(5) file.
36
37       public-inbox-fetch(1) may be used to keep a single "INBOX_DIR" up-to-
38       date.
39
40       For v2 inboxes, it will create a "$INBOX_DIR/manifest.js.gz" file to
41       speed up subsequent public-inbox-fetch(1).
42

OPTIONS

44       --epoch=RANGE
45           Restrict clones of public-inbox-v2-format(5) inboxes to the given
46           range of epochs.  The range may be a single non-negative integer or
47           a (possibly open-ended) "LOW..HIGH" range of non-negative integers.
48           "~" may be prefixed to either (or both) integer values to represent
49           the offset from the maximum possible value.
50
51           For example, "--epoch=~0" alone clones only the latest epoch,
52           "--epoch=~2.." clones the three latest epochs.
53
54           Default: "0..~0" or 0.. or "..~0" (all epochs, all three examples
55           are equivalent)
56
57       -I PATTERN
58       --include=PATTERN
59           When cloning a top-level with multiple inboxes via manifest, only
60           clone inboxes and repositories matching a given wildcard pattern
61           (using "*?" and "[]" is supported).
62
63           This is a new option in public-inbox 2.0+
64
65       --exclude=PATTERN
66           When cloning a top-level with multiple inboxes via manifest, ignore
67           inboxes and repositories matching the given wildcard pattern.
68           Supports the same wildcards as "--include"
69
70           This is a new option in public-inbox 2.0+
71
72       --inbox-config=always|v2|v1|never
73           Whether or not to retrieve the "$INBOX/_/text/config/raw" HTTP(S)
74           endpoint when cloning.
75
76           Since we can't deduce v1 inboxes from code repositories, setting
77           this to "v2" or "never" can allow faster clones of code
78           repositories if no v1 inboxes are present.
79
80           Default: "always"
81
82           This is a new option in public-inbox 2.0+
83
84       --inbox-version=NUM
85           Force a remote public-inbox version (must be 1 or 2).  This is
86           auto-detected by default, and this option exists mainly for
87           testing.
88
89           This is a new option in public-inbox 2.0+
90
91       --objstore=DIR
92           Enables space savings when the remote "manifest.js.gz" includes
93           "forkgroup" entries as generated by grokmirror 2.x.
94
95           If "DIR" does not start with "/", "./", or "../", it is treated as
96           relative to the "DESTINATION" directory.  If only "--objstore=" is
97           specified where "DIR" is an empty string (""), then "objstore"
98           ("$DESTINATION/objstore") is the implied value of "DIR".
99
100           This is a new option in public-inbox 2.0+
101
102       --manifest=FILE
103           When incrementally updating an existing mirror, load the given
104           manifest (typically "manifest.js.gz") to speed up updates.
105
106           By default, public-inbox writes the retrieved manifest to
107           "$DESTINATION/manifest.js.gz", this directive also changes the
108           destination to the specified "FILE"
109
110           If "FILE" does not start with "/", "./", or "../", it is treated as
111           relative to the "DESTINATION" directory.  If only "--manifest=" is
112           specified where "FILE" is an empty string (""), then
113           "manifest.js.gz" ("$DESTINATION/manifest.js.gz") is the implied
114           value of "FILE".
115
116           This is a new option in public-inbox 2.0+
117
118       --remote-manifest=URL|RELATIVE_PATH
119           Use an alternate location for the remote manifest.js.gz file.  This
120           may be specified as a full absolute URL (e.g
121           "--remote-manifest=https://80x24.org/lore/pub/manifest.js.gz"), or
122           a pathname relative to the ROOT_URL (e.g
123           "--remote-manifest=pub/manifest.js.gz" when ROOT_URL is
124           "https://80x24.org/lore/"
125
126           By default, "ROOT_URL/manifest.js.gz" is used.
127
128           This is a new option in public-inbox 2.0+
129
130       --project-list=FILE
131           When cloning code repos from a manifest, generate a cgit-compatible
132           project list.
133
134           If "FILE" does not start with "/", "./", or "../", it is treated as
135           relative to the "DESTINATION" directory.  If only "--project-list="
136           is specified where "FILE" is an empty string (""), then
137           "projects.list" ("$DESTINATION/projects.list") is the implied value
138           of "FILE".
139
140           This is a new option in public-inbox 2.0+
141
142       --post-update-hook=COMMAND
143           Hooks to run after a repository is cloned or updated, "COMMAND"
144           will have the bare git repository destination given as its first
145           and only argument.
146
147           For v2 inboxes, this operates on a per-epoch basis.
148
149           May be specified multiple times to run multiple commands in the
150           order specified on the command-line.
151
152           This is a new option in public-inbox 2.0+
153
154       -p
155       --prune
156           Pass the "--prune" and "--prune-tags" flags to git-fetch(1) calls
157           on incremental clones.
158
159           This is a new option in public-inbox 2.0+
160
161       --purge
162           Deletes entire repos which no longer exist in the remote manifest,
163           or are filtered out by "--include=" or "--exclude=".
164
165           This is only useful when using "--manifest"
166
167           This is a new option in public-inbox 2.0+
168
169       --exit-code
170           Exit with 127 if no updates are done when relying on a manifest.
171           Updates include fingerprint mismatches in the manifest, new
172           symlinks, new repositories, and removed repositories from the
173           --project-list
174
175           This is a new option in public-inbox 2.0+
176
177       -k
178       --keep-going
179           Continue as much as possible after an error.
180
181           This is a new option in public-inbox 2.0+
182
183       -n
184       --dry-run
185           Show what would be done, without making any changes.
186
187           This is a new option in public-inbox 2.0+
188
189       -q
190       --quiet
191           Quiets down progress messages, also passed to git-fetch(1).
192
193       -v
194       --verbose
195           Increases verbosity, also passed to git-fetch(1).
196
197       --torsocks=auto|no|yes
198       --no-torsocks
199           Whether to wrap git(1) and curl(1) commands with torsocks(1).
200
201           Default: "auto"
202
203       -j JOBS =item --jobs=JOBS
204           The number of parallel processes to spawn at once for various
205           network operations using git(1) and/or curl(1).
206

EXAMPLES

208       To mirror the most recent epochs of dwarves and LTTng inboxes:
209             public-inbox-clone --epoch=~0 \
210                   --include='*lttng*' --include='*dwarves' \
211                   https://80x24.org/lore/ /path/to/inbox-mirror
212
213           "https://lore.kernel.org/" may be used instead of
214           "https://80x24.org/lore/"
215
216       To mirror all code repos of the sparse project:
217             public-inbox-clone --objstore= --project-list= --prune \
218                   --include='*sparse*' --inbox-config=never \
219                   --remote-manifest=https://80x24.org/lore/pub/manifest.js.gz \
220                   https://80x24.org/lore/ /path/to/code-mirror
221
222           "https://git.kernel.org/" may be used instead of
223           "https://80x24.org/lore/" and the "--remote-manifest" option can be
224           omitted.
225

CONTACT

227       Feedback welcome via plain-text mail to <mailto:meta@public-inbox.org>
228
229       The mail archives are hosted at <https://public-inbox.org/meta/> and
230       <http://4uok3hntl7oi7b4uf4rtfwefqeexfzil2w6kgk2jn5z2f764irre7byd.onion/meta/>
231

COPYRIGHT

233       Copyright all contributors <mailto:meta@public-inbox.org>
234
235       License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
236