1bup-midx(1) bup-midx(1)
2
6 bup-midx - create a multi-index (.midx) file from several .idx files
7
9 bup midx [-o outfile] <-a|-f|idxnames...>
10
12 bup midx creates a multi-index (.midx) file from one or more git pack
13 index (.idx) files.
14 Note: you should no longer need to run this command by hand. It gets
16 run automatically by bup-save(1) and similar commands.
17
19 -o, --output=filename.midx
20 use the given output filename for the .midx file. Default is
21 auto-generated.
22 -a, --auto
24 automatically generate new .midx files for any .idx files where
25 it would be appropriate.
26 -f, --force
28 force generation of a single new .midx file containing all your
29 .idx files, even if other .midx files already exist. This will
30 result in the fastest backup performance, but may take a long
31 time to run.
32 --dir=packdir
34 specify the directory containing the .idx/.midx files to work
35 with. The default is $BUP_DIR/objects/pack and $BUP_DIR/index‐
36 cache/*.
37 --max-files
39 maximum number of .idx files to open at a time. You can use
40 this if you have an especially small number of file descriptors
41 available, so that midx can complete (though possibly non-opti‐
42 mally) even if it can't open all your .idx files at once. The
43 default value of this option should be fine for most people.
44 --check
46 validate a .midx file by ensuring that all objects in its con‐
47 tained .idx files exist inside the .midx. May be useful for de‐
48 bugging.
49
51 $ bup midx -a
52 Merging 21 indexes (2278559 objects).
53 Table size: 524288 (17 bits)
54 Reading indexes: 100.00% (2278559/2278559), done.
55 midx-b66d7c9afc4396187218f2936a87b865cf342672.midx
56
58 By default, bup uses git-formatted pack files, which consist of a pack
59 file (containing objects) and an idx file (containing a sorted list of
60 object names and their offsets in the .pack file).
61 Normal idx files are convenient because it means you can use git(1) to
63 access your backup datasets. However, idx files can get slow when you
64 have a lot of very large packs (which git typically doesn't have, but
65 bup often does).
66 bup .midx files consist of a single sorted list of all the objects con‐
68 tained in all the .pack files it references. This list can be binary
69 searched in about log2(m) steps, where m is the total number of ob‐
70 jects.
71 To further speed up the search, midx files also have a variable-sized
73 fanout table that reduces the first n steps of the binary search. With
74 the help of this fanout table, bup can narrow down which page of the
75 midx file a given object id would be in (if it exists) with a single
76 lookup. Thus, typical searches will only need to swap in two pages:
77 one for the fanout table, and one for the object id.
78 midx files are most useful when creating new backups, since searching
80 for a nonexistent object in the repository necessarily requires search‐
81 ing through all the index files to ensure that it does not exist.
82 (Searching for objects that do exist can be optimized; for example,
83 consecutive objects are often stored in the same pack, so we can search
84 that one first using an MRU algorithm.)
85
87 bup-save(1), bup-margin(1), bup-memtest(1)
88
90 Part of the bup(1) suite.
91
93 Avery Pennarun <apenwarr@gmail.com>.
94Bup 0.29.2 2018-10-20 bup-midx(1)