1bup-midx(1)                                                        bup-midx(1)
2
3
4

NAME

6       bup-midx - create a multi-index (.midx) file from several .idx files
7

SYNOPSIS

9       bup midx [-o outfile] <-a|-f|idxnames...>
10

DESCRIPTION

12       bup midx  creates  a multi-index (.midx) file from one or more git pack
13       index (.idx) files.
14
15       Note: you should no longer need to run this command by hand.   It  gets
16       run automatically by bup-save(1) and similar commands.
17

OPTIONS

19       -o, --output=filename.midx
20              use  the  given  output filename for the .midx file.  Default is
21              auto-generated.
22
23       -a, --auto
24              automatically generate new .midx files for any .idx files  where
25              it would be appropriate.
26
27       -f, --force
28              force  generation of a single new .midx file containing all your
29              .idx files, even if other .midx files already exist.  This  will
30              result  in  the  fastest backup performance, but may take a long
31              time to run.
32
33       --dir=packdir
34              specify the directory containing the .idx/.midx  files  to  work
35              with.   The default is $BUP_DIR/objects/pack and $BUP_DIR/index‐
36              cache/*.
37
38       --max-files
39              maximum number of .idx files to open at a  time.   You  can  use
40              this  if you have an especially small number of file descriptors
41              available, so that midx can complete (though possibly  non-opti‐
42              mally)  even  if it can't open all your .idx files at once.  The
43              default value of this option should be fine for most people.
44
45       --check
46              validate a .midx file by ensuring that all objects in  its  con‐
47              tained .idx files exist inside the .midx.  May be useful for de‐
48              bugging.
49

EXAMPLES

51              $ bup midx -a
52              Merging 21 indexes (2278559 objects).
53              Table size: 524288 (17 bits)
54              Reading indexes: 100.00% (2278559/2278559), done.
55              midx-b66d7c9afc4396187218f2936a87b865cf342672.midx
56

DISCUSSION

58       By default, bup uses git-formatted pack files, which consist of a  pack
59       file  (containing objects) and an idx file (containing a sorted list of
60       object names and their offsets in the .pack file).
61
62       Normal idx files are convenient because it means you can use git(1)  to
63       access  your backup datasets.  However, idx files can get slow when you
64       have a lot of very large packs (which git typically doesn't  have,  but
65       bup often does).
66
67       bup .midx files consist of a single sorted list of all the objects con‐
68       tained in all the .pack files it references.  This list can  be  binary
69       searched  in  about  log2(m)  steps, where m is the total number of ob‐
70       jects.
71
72       To further speed up the search, midx files also have  a  variable-sized
73       fanout table that reduces the first n steps of the binary search.  With
74       the help of this fanout table, bup can narrow down which  page  of  the
75       midx  file  a  given object id would be in (if it exists) with a single
76       lookup.  Thus, typical searches will only need to swap  in  two  pages:
77       one for the fanout table, and one for the object id.
78
79       midx  files  are most useful when creating new backups, since searching
80       for a nonexistent object in the repository necessarily requires search‐
81       ing  through  all  the  index  files  to ensure that it does not exist.
82       (Searching for objects that do exist can  be  optimized;  for  example,
83       consecutive objects are often stored in the same pack, so we can search
84       that one first using an MRU algorithm.)
85

SEE ALSO

87       bup-save(1), bup-margin(1), bup-memtest(1)
88

BUP

90       Part of the bup(1) suite.
91

AUTHORS

93       Avery Pennarun <apenwarr@gmail.com>.
94
95
96
97Bup 0.29.2                        2018-10-20                       bup-midx(1)
Impressum