1bup-memtest(1) bup-memtest(1)
2
6 bup-memtest - test bup memory usage statistics
7
9 bup memtest [options...]
10
12 bup memtest opens the list of pack indexes in your bup repository, then
13 searches the list for a series of nonexistent objects, printing memory
14 usage statistics after each cycle.
15 Because of the way Unix systems work, the output will usually show a
17 large (and unchanging) value in the VmSize column, because mapping the
18 index files in the first place takes a certain amount of virtual ad‐
19 dress space. However, this virtual memory usage is entirely virtual;
20 it doesn't take any of your RAM. Over time, bup uses parts of the in‐
21 dexes, which need to be loaded from disk, and this is what causes an
22 increase in the VmRSS column.
23
25 -n, --number=number
26 set the number of objects to search for during each cycle (ie.
27 before printing a line of output)
28 -c, --cycles=cycles
30 set the number of cycles (ie. the number of lines of output af‐
31 ter the first). The first line of output is always 0 (ie. the
32 baseline before searching for any objects).
33 --ignore-midx
35 ignore any .midx files created by bup midx. This allows you to
36 compare memory performance with and without using midx.
37 --existing
39 search for existing objects instead of searching for random
40 nonexistent ones. This can greatly affect memory usage and per‐
41 formance. Note that most of the time, bup save spends most of
42 its time searching for nonexistent objects, since existing ones
43 are probably in unmodified files that we won't be trying to back
44 up anyway. So the default behaviour reflects real bup perfor‐
45 mance more accurately. But you might want this option anyway
46 just to make sure you haven't made searching for existing ob‐
47 jects much worse than before.
48
50 $ bup memtest -n300 -c5
51 PackIdxList: using 1 index.
52 VmSize VmRSS VmData VmStk
53 0 20824 kB 4528 kB 1980 kB 84 kB
54 300 20828 kB 5828 kB 1984 kB 84 kB
55 600 20828 kB 6844 kB 1984 kB 84 kB
56 900 20828 kB 7836 kB 1984 kB 84 kB
57 1200 20828 kB 8736 kB 1984 kB 84 kB
58 1500 20828 kB 9452 kB 1984 kB 84 kB
59 $ bup memtest -n300 -c5 --ignore-midx
61 PackIdxList: using 361 indexes.
62 VmSize VmRSS VmData VmStk
63 0 27444 kB 6552 kB 2516 kB 84 kB
64 300 27448 kB 15832 kB 2520 kB 84 kB
65 600 27448 kB 17220 kB 2520 kB 84 kB
66 900 27448 kB 18012 kB 2520 kB 84 kB
67 1200 27448 kB 18388 kB 2520 kB 84 kB
68 1500 27448 kB 18556 kB 2520 kB 84 kB
69
71 When optimizing bup indexing, the first goal is to keep the VmRSS rea‐
72 sonably low. However, it might eventually be necessary to swap in all
73 the indexes, simply because you're searching for a lot of objects, and
74 this will cause your RSS to grow as large as VmSize eventually.
75 The key word here is eventually. As long as VmRSS grows reasonably
77 slowly, the amount of disk activity caused by accessing pack indexes is
78 reasonably small. If it grows quickly, bup will probably spend most of
79 its time swapping index data from disk instead of actually running your
80 backup, so backups will run very slowly.
81 The purpose of bup memtest is to give you an idea of how fast your mem‐
83 ory usage is growing, and to help in optimizing bup for better memory
84 use. If you have memory problems you might be asked to send the output
85 of bup memtest to help diagnose the problems.
86 Tip: try using bup midx -a or bup midx -f to see if it helps reduce
88 your memory usage.
89 Trivia: index memory usage in bup (or git) is only really a problem
91 when adding a large number of previously unseen objects. This is be‐
92 cause for each object, we need to absolutely confirm that it isn't al‐
93 ready in the database, which requires us to search through all the ex‐
94 isting pack indexes to ensure that none of them contain the object in
95 question. In the more obvious case of searching for objects that do
96 exist, the objects being searched for are typically related in some
97 way, which means they probably all exist in a small number of pack‐
98 files, so memory usage will be constrained to just those packfile in‐
99 dexes.
100 Since git users typically don't add a lot of files in a single run, git
102 doesn't really need a program like bup midx. bup, on the other hand,
103 spends most of its time backing up files it hasn't seen before, so its
104 memory usage patterns are different.
105
107 bup-midx(1)
108
110 Part of the bup(1) suite.
111
113 Avery Pennarun <apenwarr@gmail.com>.
114Bup 0.29.2 2018-10-20 bup-memtest(1)