1BUFFER(1) General Commands Manual BUFFER(1)
2
6 buffer - very fast reblocking program
7
9 buffer [-S size] [-b blocks] [-s size] [-m size] [-p percentage] [-u
10 microseconds] [-B] [-t] [-Z] [-i filename] [-o filename]
11
13 -i filename
14 Use the given file as the input file. The default is stdin.
15 -o filename
17 Use the given file as the output file. The default is stdout.
18 -S size
20 After every chunk of this size has been written, print out how
21 much has been written so far. Also prints the total througput. By
22 default this is not set.
23 -s size
25 Size in bytes of each block. The default blocksize is 10k to
26 match the normal output of the tar(1) program.
27 -z size
29 Combines the -S and -s flags.
30 -b blocks
32 Number of blocks to allocate to shared memory circular buffer.
33 Defaults to the number required to fill up the shared memory
34 requested.
35 -m size
37 Maximum size of the shared memory chunk to allocate for the circu‐
38 lar queue. Defaults to one megabyte.
39 -p percentage
41 Only start a write when the given percentage of the internal queue
42 is full. A percentage around 75 often proves best. Defaults to
43 zero.
44 -u microseconds
46 After every write pause for this many microseconds. Defaults to
47 zero. (Suprisingly a small sleep, 100 usecs, after each write can
48 greatly enhance throughput on some drives.)
49 -B Force each block written to be padded out to the blocksize. This
51 is needed by some tape and cartridge drives. Defaults to
52 unpadded. This only affects the last block written.
53 -t On exiting print to stderr a brief message showing the total num‐
55 ber of bytes written.
56 -Z If reading/writing directly to a character device (like a tape
58 drive) then after each gigabyte perform an lseek to the start of
59 the file. Use this flag with extreme care. It can only be used
60 on devices where an lseek does not rewind the tape but does reset
61 the kernels position flags. It is used to allow more than 2 giga‐
62 bytes to be written.
63 Sizes are a number with an optional trailing character. A 'b' multi‐
65 plies the size by 512, a 'k' by 1024 and an 'm' by a meg.
66
68 Buffer reads from standard input reblocking to the given blocksize and
69 writes each block to standard output.
70 Internally buffer is a pair of processes communicating via a large cir‐
72 cular queue held in shared memory. The reader process only has to
73 block when the queue is full and the writer process when the queue is
74 empty. Buffer is designed to try and keep the writer side continuously
75 busy so that it can stream when writing to tape drives. When used to
76 write tapes with an intervening network buffer can result in a consid‐
77 erable increase in throughput.
78 The default settings for buffer are normally good enough. If you are a
80 heavy tape user then it is worth your while trying out various differ‐
81 ent combinations of options. In particular running a buffer at both
82 ends of the pipe can provide a substantial increase (see last example
83 below).
84
86 $ buffer < /etc/termcap > /dev/rst8
87 $ tar cf - . | rsh somehost 'buffer > /dev/rst8'
89 $ dump fu - | rsh somehost 'buffer -s 16k > /dev/nrst8'
91 $ tar cf - . | buffer |
92 rsh somehost 'buffer -S 500K -p 75 > /dev/rst0'
93
95 Internally, for printing purposes, buffer counts in terms of the number
96 of kilobytes output. If the blocksize you use is not a whole number of
97 kilobytes then the numbers printed will be inaccurate.
98
101 Thanks to Kevin Twidle <kpt@doc.ic.ac.uk> for a lot of early sugges‐
102 tions and patches to make it work with non-tar/dump tapes to exabyte
103 drives.
104 Thanks to Andi Karrer <karrer@bernina.ethz.ch>, Rumi Zahir
106 <rumi@iis.ethz.ch> and Christoph Wicki <wicki@iis.ethz.ch> for patches
107 to make buffer work when trying to write single tape files of greater
108 than 2 gigabytes.
109
112 Copyright (C) 1990, 1991 by Lee McLoughlin.
113
115 dd(1), tar(1), rsh(1)
116 14 May 1990 BUFFER(1)