1bup-memtest(1)                                                  bup-memtest(1)
2
3
4

NAME

6       bup-memtest - test bup memory usage statistics
7

SYNOPSIS

9       bup memtest [options...]
10

DESCRIPTION

12       bup memtest opens the list of pack indexes in your bup repository, then
13       searches the list for a series of nonexistent objects, printing  memory
14       usage statistics after each cycle.
15
16       Because  of  the  way Unix systems work, the output will usually show a
17       large (and unchanging) value in the VmSize column, because mapping  the
18       index  files  in  the first place takes a certain amount of virtual ad‐
19       dress space.  However, this virtual memory usage is  entirely  virtual;
20       it  doesn't take any of your RAM.  Over time, bup uses parts of the in‐
21       dexes, which need to be loaded from disk, and this is  what  causes  an
22       increase in the VmRSS column.
23

OPTIONS

25       -n, --number=number
26              set  the  number of objects to search for during each cycle (ie.
27              before printing a line of output)
28
29       -c, --cycles=cycles
30              set the number of cycles (ie.  the number of lines of output af‐
31              ter  the first).  The first line of output is always 0 (ie.  the
32              baseline before searching for any objects).
33
34       --ignore-midx
35              ignore any .midx files created by bup midx.  This allows you  to
36              compare memory performance with and without using midx.
37
38       --existing
39              search  for  existing  objects  instead  of searching for random
40              nonexistent ones.  This can greatly affect memory usage and per‐
41              formance.   Note  that most of the time, bup save spends most of
42              its time searching for nonexistent objects, since existing  ones
43              are probably in unmodified files that we won't be trying to back
44              up anyway.  So the default behaviour reflects real  bup  perfor‐
45              mance  more  accurately.   But you might want this option anyway
46              just to make sure you haven't made searching  for  existing  ob‐
47              jects much worse than before.
48

EXAMPLES

50              $ bup memtest -n300 -c5
51              PackIdxList: using 1 index.
52                             VmSize      VmRSS     VmData      VmStk
53                      0    20824 kB    4528 kB    1980 kB      84 kB
54                    300    20828 kB    5828 kB    1984 kB      84 kB
55                    600    20828 kB    6844 kB    1984 kB      84 kB
56                    900    20828 kB    7836 kB    1984 kB      84 kB
57                   1200    20828 kB    8736 kB    1984 kB      84 kB
58                   1500    20828 kB    9452 kB    1984 kB      84 kB
59
60              $ bup memtest -n300 -c5 --ignore-midx
61              PackIdxList: using 361 indexes.
62                             VmSize      VmRSS     VmData      VmStk
63                      0    27444 kB    6552 kB    2516 kB      84 kB
64                    300    27448 kB   15832 kB    2520 kB      84 kB
65                    600    27448 kB   17220 kB    2520 kB      84 kB
66                    900    27448 kB   18012 kB    2520 kB      84 kB
67                   1200    27448 kB   18388 kB    2520 kB      84 kB
68                   1500    27448 kB   18556 kB    2520 kB      84 kB
69

DISCUSSION

71       When  optimizing bup indexing, the first goal is to keep the VmRSS rea‐
72       sonably low.  However, it might eventually be necessary to swap in  all
73       the  indexes, simply because you're searching for a lot of objects, and
74       this will cause your RSS to grow as large as VmSize eventually.
75
76       The key word here is eventually.  As long  as  VmRSS  grows  reasonably
77       slowly, the amount of disk activity caused by accessing pack indexes is
78       reasonably small.  If it grows quickly, bup will probably spend most of
79       its time swapping index data from disk instead of actually running your
80       backup, so backups will run very slowly.
81
82       The purpose of bup memtest is to give you an idea of how fast your mem‐
83       ory  usage  is growing, and to help in optimizing bup for better memory
84       use.  If you have memory problems you might be asked to send the output
85       of bup memtest to help diagnose the problems.
86
87       Tip:  try  using  bup midx -a  or bup midx -f to see if it helps reduce
88       your memory usage.
89
90       Trivia: index memory usage in bup (or git) is  only  really  a  problem
91       when  adding  a large number of previously unseen objects.  This is be‐
92       cause for each object, we need to absolutely confirm that it isn't  al‐
93       ready  in the database, which requires us to search through all the ex‐
94       isting pack indexes to ensure that none of them contain the  object  in
95       question.   In  the  more obvious case of searching for objects that do
96       exist, the objects being searched for are  typically  related  in  some
97       way,  which  means  they  probably all exist in a small number of pack‐
98       files, so memory usage will be constrained to just those  packfile  in‐
99       dexes.
100
101       Since git users typically don't add a lot of files in a single run, git
102       doesn't really need a program like bup midx.  bup, on the  other  hand,
103       spends  most of its time backing up files it hasn't seen before, so its
104       memory usage patterns are different.
105

SEE ALSO

107       bup-midx(1)
108

BUP

110       Part of the bup(1) suite.
111

AUTHORS

113       Avery Pennarun <apenwarr@gmail.com>.
114
115
116
117Bup 0.29.1                        2017-03-26                    bup-memtest(1)
Impressum