1lrzip(1) lrzip(1)
2
6 lrzip - a large-file compression program
7
9 lrzip [OPTIONS] <file>
10 lrzip -d [OPTIONS] <file>
11 lrunzip [OPTIONS] <file>
12 lrztar [lrzip options] <directory>
13 lrztar -d [lrzip options] <directory>
14 lrzuntar [lrzip options] <directory>
15 LRZIP=NOCONFIG [lrzip|lrunzip] [OPTIONS] <file>
16
18 LRZIP is a file compression program designed to do particularly well on
19 very large files containing long distance redundancy. lrztar is a
20 wrapper for LRZIP to simplify compression and decompression of directo‐
21 ries.
22
24 Here is a summary of the options to lrzip.
25 -w size compression window in hundreds of MB
29 default chosen by heuristic dependent on ram and chosen compression
30 -d decompress
31 -o filename specify the output file name and/or path
32 -O directory specify the output directory when -o is not used
33 -S suffix specify compressed suffix (default '.lrz')
34 -f force overwrite of any existing files
35 -D delete existing files
36 -P don't set permissions on output file. It may leave it world-readable
37 -q don't show compression progress
38 -L level set lzma/bzip2/gzip compression level (1-9, default 7)
39 -n no backend compression. Prepare for other compressor
40 -l lzo compression (ultra fast)
41 -b bzip2 compression
42 -g gzip compression using zlib
43 -z zpaq compression (best, extreme compression, extremely slow)
44 -M Maximum window and level - (all available ram and level 9)
45 -T value Compression threshold with LZO test. (0 (nil) - 10 (high), default 1)
46 -N value Set nice value to value (default 19)
47 -v[v] Increase verbosity
48 -V show version
49 -t test compressed file integrity
50 -i show compressed file information
51 If no filenames or "-" is specified, stdin/out will be used (stdin/out is inefficient with lrzip and not recommended usage).
53
58 -h Print an options summary page
59 -V Print the lrzip version number
61 -v[v] Increases verbosity. -vv will print more messages than -v.
63 -w n Set the compression window size to n in hundreds of megabytes.
65 This is the amount of memory lrzip will search during its first
66 stage of pre-compression and is the main thing that will deter‐
67 mine how much benefit lrzip will provide over ordinary compres‐
68 sion with the 2nd stage algorithm. Because of buffers and com‐
69 pression overheads, the value chosen must be significantly
70 smaller than your available ram or lrzip will induce a massive
71 swap load. If not set (recommended), the value chosen will be
72 determined by internal heuristic in lrzip which uses the most
73 memory that is reasonable. It is limited to 2GB on 32bit
74 machines.
75 -L 1..9
77 Set the compression level from 1 to 9. The default is to use
78 level 7, which is a reasonable compromise between speed and com‐
79 pression. The compression level is also strongly related to how
80 much memory lrzip uses. See the -w option for details.
81 -M Maximum compression. If this option is set, then lrzip ignores
83 the heuristic mentioned for the default window and tries to set
84 it to all available ram, and sets the compression level to maxi‐
85 mum. This will cause a significant swap load on most machines,
86 and may even fail without enough swap space allocated. Be pre‐
87 pared to walk away if you use this option. It is not recommended
88 to use this as it hardly ever improves compression.
89 -T 0..10
91 Sets the LZO compression threshold when testing a data chunk
92 when slower compression is used. The threshold level can be from
93 0 to 10. This option is used to speed up compression by avoid‐
94 ing doing the slow compression pass. The reasoning is that if it
95 is completely incompressible by LZO then it will also be incom‐
96 pressible by them, thereby saving time. The default is 1.
97 -d Decompress. If this option is not used then lrzip looks at the
99 name used to launch the program. If it contains the string
100 "lrunzip" then the -d option is automatically set.
101 -l LZO Compression. If this option is set then lrzip will use the
103 ultra fast lzo compression algorithm for the 2nd stage. This
104 mode of compression gives bzip2 like compression at the speed it
105 would normally take to simply copy the file, giving excellent
106 compression/time value]&.
107 -n No 2nd stage compression. If this option is set then lrzip will
109 only perform the long distance redundancy 1st stage compression.
110 While this does not compress any faster than LZO compression, it
111 produces a smaller file that then responds better to further
112 compression (by eg another application), also reducing the com‐
113 pression time substantially.
114 -b Bzip2 compression. Uses bzip2 compression for the 2nd stage,
116 much like the original rzip does.
117 -g Gzip compression. Uses gzip compression for the 2nd stage, much
119 like the original rzip does. Uses libz compress and uncompress
120 functions.
121 -z ZPAQ compression. Uses ZPAQ compression which is from the PAQ
123 family of compressors known for having some of the highest com‐
124 pression ratios possible but at the cost of being extremely slow
125 on both compress and decompress.
126 -o Set the output file name. If this option is not set then the
128 output file name is chosen based on the input name and the suf‐
129 fix. The -o option cannot be used if more than one file name is
130 specified on the command line.
131 -O Set the output directory for the default filename. This option
133 cannot be combined with -o.
134 -S Set the compression suffix. The default is '.lrz'.
136 -f If this option is not specified (Default) then lrzip will not
138 overwrite any existing files. If you set this option then rzip
139 will silently overwrite any files as needed.
140 -D If this option is specified then lrzip will delete the source
142 file after successful compression or decompression. When this
143 option is not specified then the source files are not deleted.
144 -P If this option is specified then lrzip will not try to set the
146 file permissions on writing the file. This helps when writing to
147 a brain damaged filesystem like fat32 on windows.
148 -q If this option is specified then lrzip will not show the per‐
150 centage progress while compressing. Note that compression hap‐
151 pens in bursts with lzma compression which is the default com‐
152 pression. This means that it will progress very rapidly for
153 short periods and then stop for long periods.
154 -N value
156 The default nice value is 19. This option can be used to set the
157 priority scheduling for the lrzip backup or decompression. Valid
158 nice values are from -20 to 19.
159 -t This tests the compressed file integrity. It does this by decom‐
161 pressing it to a temporary file and then deleting it.
162 -i This shows information about a compressed file. It shows the
164 compressed size, the decompressed size, the compression ratio
165 and what compression was used. Note that the compression mode
166 is detected from the first block only and it will show no com‐
167 pression used if the first block was incompressible, even if
168 later blocks were compressible.
169
171 "make install" or just install lrzip somewhere in your search path.
172
174 LRZIP operates in two stages. The first stage finds and encodes large
175 chunks of duplicated data over potentially very long distances (limited
176 only by your available ram) in the input file. The second stage is to
177 use a compression algorithm to compress the output of the first stage.
178 The compression algorithm can be chosen to be optimised for size (lzma
179 - default), speed (lzo), legacy (bzip2) or (gzip) or can be omitted
180 entirely doing only the first stage. A one stage only compressed file
181 can almost always improve both the compression size and speed done by a
182 subsequent compression program.
183 The key difference between lrzip and other well known compression algo‐
186 rithms is its ability to take advantage of very long distance redun‐
187 dancy. The well known deflate algorithm used in gzip uses a maximum
188 history buffer of 32k. The block sorting algorithm used in bzip2 is
189 limited to 900k of history. The history buffer in lrzip can be any size
190 long, limited only by available ram.
191 It is quite common these days to need to compress files that contain
193 long distance redundancies. For example, when compressing a set of home
194 directories several users might have copies of the same file, or of
195 quite similar files. It is also common to have a single file that con‐
196 tains large duplicated chunks over long distances, such as pdf files
197 containing repeated copies of the same image. Most compression programs
198 won't be able to take advantage of this redundancy, and thus might
199 achieve a much lower compression ratio than lrzip can achieve.
200
202 LRZIP now recognizes a configuration file that contains default set‐
203 tings. This configuration is searched for in the current directory,
204 /etc/lrzip, and $HOME/.lrzip. The configuration filename must be
205 lrzip.conf.
206
208 By default, lrzip will search for and use a configuration file,
209 lrzip.conf. If the user wishes to bypass the file, a startup ENV vari‐
210 able may be set.
211 LRZIP = NOCONFIG [lrzip|lrunzip] [OPTIONS] <file>
212 which will force lrzip to ignore the configuration file.
213
215 The ideas behind rzip were first implemented in 1998 while I was work‐
216 ing on rsync. That version was too slow to be practical, and was
217 replaced by this version in 2003. LRZIP was created by the desire to
218 have better compression and/or speed by Con Kolivas on blending the
219 lzma and lzo compression algorithms with the rzip first stage, and
220 extending the compression windows to scale with increasing ram sizes.
221
223 Nil known. Probably lots.
224
227 lrzip.conf(5), bzip2(1), gzip(1), lzop(1), lrzip(1), rzip(1), zip(1)
228 lrztar(1), lrzuntar(1)
229
232 rzip was written by Andrew Tridgell.
233 lzma was written by Igor Pavlov.
234 lzo was written by Markus Oberhumer.
235 zpaq was written by Matt Mahoney.
236 lrzip was bastardised from rzip by Con Kolivas.
237 Peter Hyman added informational output, updated LZMA SDK, and aded
238 multi-threading capabilities.
239 If you wish to report a problem or make a suggestion then please email
241 Con at kernel@kolivas.org
242 lrzip is released under the GNU General Public License version 2.
244 Please see the file COPYING for license details.
245 May 2010 lrzip(1)