1cstream(1)                BSD General Commands Manual               cstream(1)
2

NAME

4     cstream — direct data streams, with bandwidth limiting, FIFO, audio,
5     duplication and extended reporting support.
6

SYNOPSIS

8     cstream [-b num] [-B num] [-i filename] [-I string] [-l] [-n num]
9             [-o filename] [-O string] [-p filename] [-t num] [-T num]
10             [-v num] [-V] [filename]
11

DESCRIPTION

13     Cstream filters data streams, much like the UNIX tool dd(1).  It has a
14     more traditional commandline syntax, support for precise bandwidth limit‐
15     ing and reporting and support for FIFOs. Data limits and throughput rate
16     calculation will work for files > 4 GB.
17
18     Cstream reads from the standard input and writes to the standard output,
19     if no filenames are given. It will also 'generate' or 'sink' data if
20     desired.
21
22     Options:
23
24     -b num    Set the block size used for read/write to num.  The default is
25               8192 bytes.
26               FIXME: The buffer size logic inside cstream is broken.
27
28     -B num    Buffer input up to num bytes before writing. The default is the
29               blocksize. It is an error to set this to anything below the
30               blocksize. Useful when writing tapes and simlilar that prefer
31               few large writes of many small.
32               FIXME: The buffer size logic inside cstream is broken.
33
34     -c num    Concurrent operation. Use a seperate process for outout. This
35               is especially useful in combination with the -B option.
36               0 = use one process only (default)
37               1 = read process will buffer
38               2 = write process will buffer
39               3 = both processes will buffer.
40                   In combination with a large buffer size this will often
41                   load your memory heavily, everytime the reader transfers
42                   the buffer it collected to the writer. If you use -c 3 and
43                   have a buffer size of 128 Megabytes 256 MB of memory will
44                   be touched at once.
45
46     -i num
47
48     -o num    Set the file names to use for input or output, respectivly. If
49               the output file name is "-", data will just be discarded. If
50               the input file name is "-", data will be generated 'out of the
51               void'. If these options aren't given, stdin/stout will be used.
52               If you need to give -o or -i options and want stdin/stdout,
53               specify the empty string, like this:
54
55               cstream -i''
56
57               If TCP support has been compiled in (default), hostname:port‐
58               number will try to connect to the specified host at the speci‐
59               fied port and :portnumber will open a TCP socket on the local
60               machine and wait for a connection to arrive. SECURITY NOTE:
61               cstream includes no mechanism to restrict the hosts that may
62               connect to this port. Unless your machine has other network
63               filters, anyone will be able to connect.
64
65     -I string
66
67     -O string
68               Specify the type of input and output file, respectivly.
69               If string
70                   includes 'f', a fifo will be created.
71               If string
72                   includes 'a', the file will be assumed to be a opensound-
73                   compatible audio device and will be switched to CD-like
74                   settings.
75               If string
76                   includes 't', a copy of the stream will be sent to file
77                   descriptor 3.
78               If string
79                   includes 'N', TCP will not be used for that file even if
80                   the name has a ":".
81
82     -l        Include line count in statistics.
83
84     -n num    Limit the total amount of data to num.  If there is more input
85               available, it will be discarded, cstream will exit after the
86               limit has been reached. If there is less input, the limit will
87               not be reached and no error will be signaled.
88
89               num may have a trailing 'k', 'm' or 'g' which means Kilobytes,
90               Megabytes or Gigabytes (where Kilo = 1024). This applies to all
91               numeric options.
92
93     -p filename
94               Write the process id of cstream to filename.  If cstream uses a
95               seperate writer process (option -c), this is the pid of the
96               parent (reader) process.
97
98     -t num    Limit the throughput of the data stream to num bytes/second.
99               Limiting is done at the input side, you can rely on cstream not
100               accepting more than this rate. If the number you give is posi‐
101               tive, cstream accumulates errors and tries to keep the overall
102               rate at the specified value, for the whole session. If you give
103               a negative number, it is an upper limit for each read/write
104               system call pair. In other words: the negative number will
105               never exceed that limit, the positive number will exceed it to
106               make good for previous underutilization.
107
108     -T num    Report throughput every num seconds.
109
110     -v num    Set verbose level to num.  By default, it is set to 0, which
111               means no messages are displayed as long as no errors occur. A
112               value of 1 means that total amount of data and throughput will
113               be displayed at the end of program run. A value of 2 means the
114               transfer rate since the end of the first read/write pair will
115               also be reported (useful when there is an initial delay). A
116               value of 3 means there will also be seperate measurements for
117               read and write. This option is resource-consuming and currently
118               isn't implemented. A value of 4 means that notices about each
119               single read/write will be displayed. High values include all
120               message types of lower values.
121
122     -V        Print version number to stdout and exit with 0.
123
124     filename  A single filename as the last argument without an option switch
125               will be used as input file if -i has not been used.
126
127     SIGUSR1
128
129     SIGINFO   Sending SIGUSR1 (or SIGINFO, which is usually mappend to Con‐
130               trol-T on you keyboard) to cstream causes it to display
131               throughput rates to stderr. The stream will continue as if
132               nothing happend.
133
134     SIGUSR2   Exit and report throughput rates, if requested.
135
136     SIGHUP    I found myself sending SIGHUP accidentially too often. But
137               ignoring or misusing SIGHUP is not an option for me. Thus, when
138               cstream received SIGHUP, it will wait 5 seconds for another
139               SIGHUP, to give users a chance to correct a possible mistake.
140               If no additional SIGHUP is received, cstream kills itself with
141               SIGHUP.
142

EXAMPLES

144     cstream -o tmpfile -v 1 -n 384m -i -
145             Writes 384 Megabytes of unspecified data to file tmpfile and dis‐
146             play verbose throughput rate. Makes a good benchmark, the speed
147             of /dev/null varies too much from system to system.
148
149     cstream -i tmpfile -v 1 -n 384m -o -
150             Read the same file back in and discard data.
151
152     cstream -b 2000  -t 10000 /var/log/messages
153             Will display the file in a more or less watchable speed.
154
155     dump 0sf 400000 - / | cstream -v 1 -b 32768 -o /dev/rst0 -p pidfile
156
157     kill -USR1 `cat pidfile`
158             Write the output from dump(1) to tape. Each time the signal is
159             sent, the throughput and data rate so far will be displayed.
160
161     cstream -t 176400 -i /dev/dsp0 -I f -o -
162             Makes kind of a soundcard emulator which may be used to test
163             audio applications that need something to write to that limits
164             the data rate as a real soundcard does. This obviously doesn't
165             work when the application tries to write data using mmap(2) and
166             the application has to ignore errors when it tries to set sound‐
167             card parameters using ioctl(2).
168
169     cstream -t 176400 -i /dev/dsp0 -I f -o /dev/dsp1 -O f
170             Similar soundcard emulator, except that it allows you to grab the
171             data your applications sends to it from the other fifo, while
172             still having precise timing.
173
174     cstream -Oa -o /dev/dsp0 myhost.mydomain.com:17324
175             Connects port 3333 on host myhost.mydomain.com and whatever data
176             it finds there will be sent to the soundcard, with appropriate
177             settings for CD quality stero play.
178
179     cstream -i myaudiofile.raw -o :17324
180             This will open a TCP server on port 17324 and waits until someone
181             connects (for example, the commandline from the previous exam‐
182             ple). Then it will send the contents of myaudiofile.raw down the
183             TCP stream (for the previous audio example, typically a CD audio‐
184             track like you get from the tosha or cdparanoia utilities).
185
186     cstream -OD -o myfile
187
188             Write to file myfile with O_DIRECT.  That usually means that the
189             filesystem buffer cache will not try to cache this file.  You can
190             use that to prevent copying operations from eating up physical
191             memory.  Note that when cstream encouters a write error it will
192             switch the output file from O_DIRECT to a normal file and write
193             all further blocks without O_DIRECT if writes without O_DIRECT
194             succeed.  In practice that usually means that your last block, if
195             not a multiple of the filesystem block size, will still be writ‐
196             ten into the file (the maximum amount of data written without
197             O_DIRECT is your blocksize minus one).  That way cstream ensures
198             that the output file has the length of the input, however odd the
199             length was and no matter what restrictions your OS places on
200             O_DIRECT output.  Again, cstream will *not* pad the output to the
201             block size, you get the same file and file size as if not using
202             O_DIRECT, at the cost of switching to non-O_DIRECT whenever a
203             block is not the right size.
204
205     cstream -i :3333 | dd obs=8192 | ./cstream -omyfile -v7 -OD
206             This is what you need to do to buffer TCP input, so that the last
207             cstream will not switch away from O_DIRECT prematurely because of
208             short reads.  If your input can do short reads (e.g. from TCP),
209             and you want to ensure that O_DIRECT stays in effect, you need a
210             buffer between the TCP stream and the O_DIRECT stream.  Since
211             cstream does not yet support different input and output block
212             sizes, dd is suitable here.  Note that this is only neccessary if
213             the OS requires multiples of the filesystem block size for
214             O_DIRECT.  At the time of this writing this construct is needed
215             on Linux for using TCP streams with O_DIRECT, but it is not
216             needed on FreeBSD.
217
218     cstream -OS -o myfile
219             Writes to file myfile with O_SYNC.  This means by the time the
220             system call returns the data is known to be on disk.  This is not
221             the same thing as O_DIRECT.  O_DIRECT can do its own buffering,
222             with O_SYNC there is no buffering at all.  At the time of this
223             writing, O_SYNC on both Linux and FreeBSD is very slow (1/5th to
224             1/10th of normal write) and O_DIRECT is reasonably fast (1/4th to
225             1/2 of normal write).  You can combined O_SYNC and O_DIRECT.
226

ERRORS

228     Exit code 0 means success.
229
230     Exit code 1 means a commandline syntax usage error.
231
232     Exit code 2 means other errors, especially system errors.
233

Bugs

235     There should be an option to begin writing directly after the first read
236     ended and then fill the buffer with reads in the background.  Right now
237     writing will not begin before the reader has filled the buffer completely
238     for the first time.
239
240     Not a bug: the code to do O_DIRECT is reasonably sophisticated.  It will
241     fall back to normal I/O on errors.  But before doing that it knows about
242     both filesystem blocksize requirements (will default I/O blocksize to
243     whatever the filesystem of the output file is in) and page alignment
244     requirements (I/O will happen from a page-aligned buffer).  However, the
245     combination of concurrent read/writes (-c options) and O_DIRECT has not
246     been tested bejond basic verification that it gets some tests right.
247

SEE ALSO

249     dd(1), mkfifo(2)
250

HISTORY

252     cstream was initially written by Martin Cracauer in 1998.  For updates
253     and more information see http://www.cons.org/cracauer/cstream.html
254
255BSD                             March, 30, 1999                            BSD
Impressum