1GIT-PACK-OBJECTS(1) Git Manual GIT-PACK-OBJECTS(1)
2
6 git-pack-objects - Create a packed archive of objects
7
9 git pack-objects [-q | --progress | --all-progress] [--all-progress-implied]
10 [--no-reuse-delta] [--delta-base-offset] [--non-empty]
11 [--local] [--incremental] [--window=<n>] [--depth=<n>]
12 [--revs [--unpacked | --all]] [--keep-pack=<pack-name>]
13 [--cruft] [--cruft-expiration=<time>]
14 [--stdout [--filter=<filter-spec>] | <base-name>]
15 [--shallow] [--keep-true-parents] [--[no-]sparse] < <object-list>
16
18 Reads list of objects from the standard input, and writes either one or
19 more packed archives with the specified base-name to disk, or a packed
20 archive to the standard output.
21 A packed archive is an efficient way to transfer a set of objects
23 between two repositories as well as an access efficient archival
24 format. In a packed archive, an object is either stored as a compressed
25 whole or as a difference from some other object. The latter is often
26 called a delta.
27 The packed archive format (.pack) is designed to be self-contained so
29 that it can be unpacked without any further information. Therefore,
30 each object that a delta depends upon must be present within the pack.
31 A pack index file (.idx) is generated for fast, random access to the
33 objects in the pack. Placing both the index file (.idx) and the packed
34 archive (.pack) in the pack/ subdirectory of $GIT_OBJECT_DIRECTORY (or
35 any of the directories on $GIT_ALTERNATE_OBJECT_DIRECTORIES) enables
36 Git to read from the pack archive.
37 The git unpack-objects command can read the packed archive and expand
39 the objects contained in the pack into "one-file one-object" format;
40 this is typically done by the smart-pull commands when a pack is
41 created on-the-fly for efficient network transport by their peers.
42
44 base-name
45 Write into pairs of files (.pack and .idx), using <base-name> to
46 determine the name of the created file. When this option is used,
47 the two files in a pair are written in
48 <base-name>-<SHA-1>.{pack,idx} files. <SHA-1> is a hash based on
49 the pack content and is written to the standard output of the
50 command.
51 --stdout
53 Write the pack contents (what would have been written to .pack
54 file) out to the standard output.
55 --revs
57 Read the revision arguments from the standard input, instead of
58 individual object names. The revision arguments are processed the
59 same way as git rev-list with the --objects flag uses its commit
60 arguments to build the list of objects it outputs. The objects on
61 the resulting list are packed. Besides revisions, --not or
62 --shallow <SHA-1> lines are also accepted.
63 --unpacked
65 This implies --revs. When processing the list of revision arguments
66 read from the standard input, limit the objects packed to those
67 that are not already packed.
68 --all
70 This implies --revs. In addition to the list of revision arguments
71 read from the standard input, pretend as if all refs under refs/
72 are specified to be included.
73 --include-tag
75 Include unasked-for annotated tags if the object they reference was
76 included in the resulting packfile. This can be useful to send new
77 tags to native Git clients.
78 --stdin-packs
80 Read the basenames of packfiles (e.g., pack-1234abcd.pack) from the
81 standard input, instead of object names or revision arguments. The
82 resulting pack contains all objects listed in the included packs
83 (those not beginning with ^), excluding any objects listed in the
84 excluded packs (beginning with ^).
85 Incompatible with --revs, or options that imply --revs (such as
87 --all), with the exception of --unpacked, which is compatible.
88 --cruft
90 Packs unreachable objects into a separate "cruft" pack, denoted by
91 the existence of a .mtimes file. Typically used by git repack
92 --cruft. Callers provide a list of pack names and indicate which
93 packs will remain in the repository, along with which packs will be
94 deleted (indicated by the - prefix). The contents of the cruft pack
95 are all objects not contained in the surviving packs which have not
96 exceeded the grace period (see --cruft-expiration below), or which
97 have exceeded the grace period, but are reachable from an other
98 object which hasn’t.
99 When the input lists a pack containing all reachable objects (and
101 lists all other packs as pending deletion), the corresponding cruft
102 pack will contain all unreachable objects (with mtime newer than
103 the --cruft-expiration) along with any unreachable objects whose
104 mtime is older than the --cruft-expiration, but are reachable from
105 an unreachable object whose mtime is newer than the
106 --cruft-expiration).
107 Incompatible with --unpack-unreachable, --keep-unreachable,
109 --pack-loose-unreachable, --stdin-packs, as well as any other
110 options which imply --revs. Also incompatible with --max-pack-size;
111 when this option is set, the maximum pack size is not inferred from
112 pack.packSizeLimit.
113 --cruft-expiration=<approxidate>
115 If specified, objects are eliminated from the cruft pack if they
116 have an mtime older than <approxidate>. If unspecified (and given
117 --cruft), then no objects are eliminated.
118 --window=<n>, --depth=<n>
120 These two options affect how the objects contained in the pack are
121 stored using delta compression. The objects are first internally
122 sorted by type, size and optionally names and compared against the
123 other objects within --window to see if using delta compression
124 saves space. --depth limits the maximum delta depth; making it too
125 deep affects the performance on the unpacker side, because delta
126 data needs to be applied that many times to get to the necessary
127 object.
128 The default value for --window is 10 and --depth is 50. The maximum
130 depth is 4095.
131 --window-memory=<n>
133 This option provides an additional limit on top of --window; the
134 window size will dynamically scale down so as to not take up more
135 than <n> bytes in memory. This is useful in repositories with a mix
136 of large and small objects to not run out of memory with a large
137 window, but still be able to take advantage of the large window for
138 the smaller objects. The size can be suffixed with "k", "m", or
139 "g". --window-memory=0 makes memory usage unlimited. The default
140 is taken from the pack.windowMemory configuration variable.
141 --max-pack-size=<n>
143 In unusual scenarios, you may not be able to create files larger
144 than a certain size on your filesystem, and this option can be used
145 to tell the command to split the output packfile into multiple
146 independent packfiles, each not larger than the given size. The
147 size can be suffixed with "k", "m", or "g". The minimum size
148 allowed is limited to 1 MiB. The default is unlimited, unless the
149 config variable pack.packSizeLimit is set. Note that this option
150 may result in a larger and slower repository; see the discussion in
151 pack.packSizeLimit.
152 --honor-pack-keep
154 This flag causes an object already in a local pack that has a .keep
155 file to be ignored, even if it would have otherwise been packed.
156 --keep-pack=<pack-name>
158 This flag causes an object already in the given pack to be ignored,
159 even if it would have otherwise been packed. <pack-name> is the
160 pack file name without leading directory (e.g. pack-123.pack). The
161 option could be specified multiple times to keep multiple packs.
162 --incremental
164 This flag causes an object already in a pack to be ignored even if
165 it would have otherwise been packed.
166 --local
168 This flag causes an object that is borrowed from an alternate
169 object store to be ignored even if it would have otherwise been
170 packed.
171 --non-empty
173 Only create a packed archive if it would contain at least one
174 object.
175 --progress
177 Progress status is reported on the standard error stream by default
178 when it is attached to a terminal, unless -q is specified. This
179 flag forces progress status even if the standard error stream is
180 not directed to a terminal.
181 --all-progress
183 When --stdout is specified then progress report is displayed during
184 the object count and compression phases but inhibited during the
185 write-out phase. The reason is that in some cases the output stream
186 is directly linked to another command which may wish to display
187 progress status of its own as it processes incoming pack data. This
188 flag is like --progress except that it forces progress report for
189 the write-out phase as well even if --stdout is used.
190 --all-progress-implied
192 This is used to imply --all-progress whenever progress display is
193 activated. Unlike --all-progress this flag doesn’t actually force
194 any progress display by itself.
195 -q
197 This flag makes the command not to report its progress on the
198 standard error stream.
199 --no-reuse-delta
201 When creating a packed archive in a repository that has existing
202 packs, the command reuses existing deltas. This sometimes results
203 in a slightly suboptimal pack. This flag tells the command not to
204 reuse existing deltas but compute them from scratch.
205 --no-reuse-object
207 This flag tells the command not to reuse existing object data at
208 all, including non deltified object, forcing recompression of
209 everything. This implies --no-reuse-delta. Useful only in the
210 obscure case where wholesale enforcement of a different compression
211 level on the packed data is desired.
212 --compression=<n>
214 Specifies compression level for newly-compressed data in the
215 generated pack. If not specified, pack compression level is
216 determined first by pack.compression, then by core.compression, and
217 defaults to -1, the zlib default, if neither is set. Add
218 --no-reuse-object if you want to force a uniform compression level
219 on all data no matter the source.
220 --[no-]sparse
222 Toggle the "sparse" algorithm to determine which objects to include
223 in the pack, when combined with the "--revs" option. This algorithm
224 only walks trees that appear in paths that introduce new objects.
225 This can have significant performance benefits when computing a
226 pack to send a small change. However, it is possible that extra
227 objects are added to the pack-file if the included commits contain
228 certain types of direct renames. If this option is not included, it
229 defaults to the value of pack.useSparse, which is true unless
230 otherwise specified.
231 --thin
233 Create a "thin" pack by omitting the common objects between a
234 sender and a receiver in order to reduce network transfer. This
235 option only makes sense in conjunction with --stdout.
236 Note: A thin pack violates the packed archive format by omitting
238 required objects and is thus unusable by Git without making it
239 self-contained. Use git index-pack --fix-thin (see git-index-
240 pack(1)) to restore the self-contained property.
241 --shallow
243 Optimize a pack that will be provided to a client with a shallow
244 repository. This option, combined with --thin, can result in a
245 smaller pack at the cost of speed.
246 --delta-base-offset
248 A packed archive can express the base object of a delta as either a
249 20-byte object name or as an offset in the stream, but ancient
250 versions of Git don’t understand the latter. By default, git
251 pack-objects only uses the former format for better compatibility.
252 This option allows the command to use the latter format for
253 compactness. Depending on the average delta chain length, this
254 option typically shrinks the resulting packfile by 3-5 per-cent.
255 Note: Porcelain commands such as git gc (see git-gc(1)), git repack
257 (see git-repack(1)) pass this option by default in modern Git when
258 they put objects in your repository into pack files. So does git
259 bundle (see git-bundle(1)) when it creates a bundle.
260 --threads=<n>
262 Specifies the number of threads to spawn when searching for best
263 delta matches. This requires that pack-objects be compiled with
264 pthreads otherwise this option is ignored with a warning. This is
265 meant to reduce packing time on multiprocessor machines. The
266 required amount of memory for the delta search window is however
267 multiplied by the number of threads. Specifying 0 will cause Git to
268 auto-detect the number of CPU’s and set the number of threads
269 accordingly.
270 --index-version=<version>[,<offset>]
272 This is intended to be used by the test suite only. It allows to
273 force the version for the generated pack index, and to force 64-bit
274 index entries on objects located above the given offset.
275 --keep-true-parents
277 With this option, parents that are hidden by grafts are packed
278 nevertheless.
279 --filter=<filter-spec>
281 Requires --stdout. Omits certain objects (usually blobs) from the
282 resulting packfile. See git-rev-list(1) for valid <filter-spec>
283 forms.
284 --no-filter
286 Turns off any previous --filter= argument.
287 --missing=<missing-action>
289 A debug option to help with future "partial clone" development.
290 This option specifies how missing objects are handled.
291 The form --missing=error requests that pack-objects stop with an
293 error if a missing object is encountered. If the repository is a
294 partial clone, an attempt to fetch missing objects will be made
295 before declaring them missing. This is the default action.
296 The form --missing=allow-any will allow object traversal to
298 continue if a missing object is encountered. No fetch of a missing
299 object will occur. Missing objects will silently be omitted from
300 the results.
301 The form --missing=allow-promisor is like allow-any, but will only
303 allow object traversal to continue for EXPECTED promisor missing
304 objects. No fetch of a missing object will occur. An unexpected
305 missing object will raise an error.
306 --exclude-promisor-objects
308 Omit objects that are known to be in the promisor remote. (This
309 option has the purpose of operating only on locally created
310 objects, so that when we repack, we still maintain a distinction
311 between locally created objects [without .promisor] and objects
312 from the promisor remote [with .promisor].) This is used with
313 partial clone.
314 --keep-unreachable
316 Objects unreachable from the refs in packs named with --unpacked=
317 option are added to the resulting pack, in addition to the
318 reachable objects that are not in packs marked with *.keep files.
319 This implies --revs.
320 --pack-loose-unreachable
322 Pack unreachable loose objects (and their loose counterparts
323 removed). This implies --revs.
324 --unpack-unreachable
326 Keep unreachable objects in loose form. This implies --revs.
327 --delta-islands
329 Restrict delta matches based on "islands". See DELTA ISLANDS below.
330
332 When possible, pack-objects tries to reuse existing on-disk deltas to
333 avoid having to search for new ones on the fly. This is an important
334 optimization for serving fetches, because it means the server can avoid
335 inflating most objects at all and just send the bytes directly from
336 disk. This optimization can’t work when an object is stored as a delta
337 against a base which the receiver does not have (and which we are not
338 already sending). In that case the server "breaks" the delta and has to
339 find a new one, which has a high CPU cost. Therefore it’s important for
340 performance that the set of objects in on-disk delta relationships
341 match what a client would fetch.
342 In a normal repository, this tends to work automatically. The objects
344 are mostly reachable from the branches and tags, and that’s what
345 clients fetch. Any deltas we find on the server are likely to be
346 between objects the client has or will have.
347 But in some repository setups, you may have several related but
349 separate groups of ref tips, with clients tending to fetch those groups
350 independently. For example, imagine that you are hosting several
351 "forks" of a repository in a single shared object store, and letting
352 clients view them as separate repositories through GIT_NAMESPACE or
353 separate repos using the alternates mechanism. A naive repack may find
354 that the optimal delta for an object is against a base that is only
355 found in another fork. But when a client fetches, they will not have
356 the base object, and we’ll have to find a new delta on the fly.
357 A similar situation may exist if you have many refs outside of
359 refs/heads/ and refs/tags/ that point to related objects (e.g.,
360 refs/pull or refs/changes used by some hosting providers). By default,
361 clients fetch only heads and tags, and deltas against objects found
362 only in those other groups cannot be sent as-is.
363 Delta islands solve this problem by allowing you to group your refs
365 into distinct "islands". Pack-objects computes which objects are
366 reachable from which islands, and refuses to make a delta from an
367 object A against a base which is not present in all of A's islands.
368 This results in slightly larger packs (because we miss some delta
369 opportunities), but guarantees that a fetch of one island will not have
370 to recompute deltas on the fly due to crossing island boundaries.
371 When repacking with delta islands the delta window tends to get clogged
373 with candidates that are forbidden by the config. Repacking with a big
374 --window helps (and doesn’t take as long as it otherwise might because
375 we can reject some object pairs based on islands before doing any
376 computation on the content).
377 Islands are configured via the pack.island option, which can be
379 specified multiple times. Each value is a left-anchored regular
380 expressions matching refnames. For example:
381 [pack]
383 island = refs/heads/
384 island = refs/tags/
385 puts heads and tags into an island (whose name is the empty string; see
387 below for more on naming). Any refs which do not match those regular
388 expressions (e.g., refs/pull/123) is not in any island. Any object
389 which is reachable only from refs/pull/ (but not heads or tags) is
390 therefore not a candidate to be used as a base for refs/heads/.
391 Refs are grouped into islands based on their "names", and two regexes
393 that produce the same name are considered to be in the same island. The
394 names are computed from the regexes by concatenating any capture groups
395 from the regex, with a - dash in between. (And if there are no capture
396 groups, then the name is the empty string, as in the above example.)
397 This allows you to create arbitrary numbers of islands. Only up to 14
398 such capture groups are supported though.
399 For example, imagine you store the refs for each fork in
401 refs/virtual/ID, where ID is a numeric identifier. You might then
402 configure:
403 [pack]
405 island = refs/virtual/([0-9]+)/heads/
406 island = refs/virtual/([0-9]+)/tags/
407 island = refs/virtual/([0-9]+)/(pull)/
408 That puts the heads and tags for each fork in their own island (named
410 "1234" or similar), and the pull refs for each go into their own
411 "1234-pull".
412 Note that we pick a single island for each regex to go into, using
414 "last one wins" ordering (which allows repo-specific config to take
415 precedence over user-wide config, and so forth).
416
418 Various configuration variables affect packing, see git-config(1)
419 (search for "pack" and "delta").
420 Notably, delta compression is not used on objects larger than the
422 core.bigFileThreshold configuration variable and on files with the
423 attribute delta set to false.
424
426 git-rev-list(1) git-repack(1) git-prune-packed(1)
427
429 Part of the git(1) suite
430Git 2.39.1 2023-01-13 GIT-PACK-OBJECTS(1)