1cstream(1) BSD General Commands Manual cstream(1)
2
4 cstream — direct data streams, with bandwidth limiting, FIFO, audio, du‐
5 plication and extended reporting support.
6
8 cstream [-b num] [-B num] [-i filename] [-I string] [-l] [-n num]
9 [-o filename] [-O string] [-p filename] [-t num] [-T num]
10 [-v num] [-V] [filename]
11
13 Cstream filters data streams, much like the UNIX tool dd(1). It has a
14 more traditional commandline syntax, support for precise bandwidth limit‐
15 ing and reporting and support for FIFOs. Data limits and throughput rate
16 calculation will work for files > 4 GB.
17 Cstream reads from the standard input and writes to the standard output,
19 if no filenames are given. It will also 'generate' or 'sink' data if de‐
20 sired.
21 Options:
23 -b num Set the block size used for read/write to num. The default is
25 8192 bytes.
26 FIXME: The buffer size logic inside cstream is broken.
27 -B num Buffer input up to num bytes before writing. The default is the
29 blocksize. It is an error to set this to anything below the
30 blocksize. Useful when writing tapes and simlilar that prefer
31 few large writes of many small.
32 FIXME: The buffer size logic inside cstream is broken.
33 -c num Concurrent operation. Use a seperate process for outout. This
35 is especially useful in combination with the -B option.
36 0 = use one process only (default)
37 1 = read process will buffer
38 2 = write process will buffer
39 3 = both processes will buffer.
40 In combination with a large buffer size this will often
41 load your memory heavily, everytime the reader transfers
42 the buffer it collected to the writer. If you use -c 3 and
43 have a buffer size of 128 Megabytes 256 MB of memory will
44 be touched at once.
45 -i num
47 -o num Set the file names to use for input or output, respectivly. If
49 the output file name is "-", data will just be discarded. If
50 the input file name is "-", data will be generated 'out of the
51 void'. If these options aren't given, stdin/stout will be used.
52 If you need to give -o or -i options and want stdin/stdout,
53 specify the empty string, like this:
54 cstream -i''
56 If TCP support has been compiled in (default), hostname:port‐
58 number will try to connect to the specified host at the speci‐
59 fied port and :portnumber will open a TCP socket on the local
60 machine and wait for a connection to arrive. SECURITY NOTE:
61 cstream includes no mechanism to restrict the hosts that may
62 connect to this port. Unless your machine has other network
63 filters, anyone will be able to connect.
64 -I string
66 -O string
68 Specify the type of input and output file, respectivly.
69 If string
70 includes 'f', a fifo will be created.
71 If string
72 includes 'F', a rsync(2) will be issued before closing the
73 output file.
74 If string
75 includes 'a', the file will be assumed to be a opensound-
76 compatible audio device and will be switched to CD-like
77 settings.
78 If string
79 includes 't', a copy of the stream will be sent to file de‐
80 scriptor 3.
81 If string
82 includes 'N', TCP will not be used for that file even if
83 the name has a ":".
84 -l Include line count in statistics.
86 -n num Limit the total amount of data to num. If there is more input
88 available, it will be discarded, cstream will exit after the
89 limit has been reached. If there is less input, the limit will
90 not be reached and no error will be signaled.
91 num may have a trailing 'k', 'm' or 'g' which means Kilobytes,
93 Megabytes or Gigabytes (where Kilo = 1024). This applies to all
94 numeric options.
95 -p filename
97 Write the process id of cstream to filename. If cstream uses a
98 seperate writer process (option -c), this is the pid of the
99 parent (reader) process.
100 -t num Limit the throughput of the data stream to num bytes/second.
102 Limiting is done at the input side, you can rely on cstream not
103 accepting more than this rate. If the number you give is posi‐
104 tive, cstream accumulates errors and tries to keep the overall
105 rate at the specified value, for the whole session. If you give
106 a negative number, it is an upper limit for each read/write
107 system call pair. In other words: the negative number will
108 never exceed that limit, the positive number will exceed it to
109 make good for previous underutilization.
110 -T num Report throughput every num seconds.
112 -v num Set verbose level to num. By default, it is set to 0, which
114 means no messages are displayed as long as no errors occur. A
115 value of 1 means that total amount of data and throughput will
116 be displayed at the end of program run. A value of 2 means the
117 transfer rate since the end of the first read/write pair will
118 also be reported (useful when there is an initial delay). A
119 value of 3 means there will also be seperate measurements for
120 read and write. This option is resource-consuming and currently
121 isn't implemented. A value of 4 means that notices about each
122 single read/write will be displayed. High values include all
123 message types of lower values.
124 -V Print version number to stdout and exit with 0.
126 filename A single filename as the last argument without an option switch
128 will be used as input file if -i has not been used.
129 SIGUSR1
131 SIGINFO Sending SIGUSR1 (or SIGINFO, which is usually mappend to Con‐
133 trol-T on you keyboard) to cstream causes it to display
134 throughput rates to stderr. The stream will continue as if
135 nothing happend.
136 SIGUSR2 Exit and report throughput rates, if requested.
138 SIGHUP I found myself sending SIGHUP accidentially too often. But ig‐
140 noring or misusing SIGHUP is not an option for me. Thus, when
141 cstream received SIGHUP, it will wait 5 seconds for another
142 SIGHUP, to give users a chance to correct a possible mistake.
143 If no additional SIGHUP is received, cstream kills itself with
144 SIGHUP.
145
147 cstream -o tmpfile -v 1 -n 384m -i -
148 Writes 384 Megabytes of unspecified data to file tmpfile and dis‐
149 play verbose throughput rate. Makes a good benchmark, the speed
150 of /dev/null varies too much from system to system.
151 cstream -i tmpfile -v 1 -n 384m -o -
153 Read the same file back in and discard data.
154 cstream -b 2000 -t 10000 /var/log/messages
156 Will display the file in a more or less watchable speed.
157 dump 0sf 400000 - / | cstream -v 1 -b 32768 -o /dev/rst0 -p pidfile
159 kill -USR1 `cat pidfile`
161 Write the output from dump(1) to tape. Each time the signal is
162 sent, the throughput and data rate so far will be displayed.
163 cstream -t 176400 -i /dev/dsp0 -I f -o -
165 Makes kind of a soundcard emulator which may be used to test au‐
166 dio applications that need something to write to that limits the
167 data rate as a real soundcard does. This obviously doesn't work
168 when the application tries to write data using mmap(2) and the
169 application has to ignore errors when it tries to set soundcard
170 parameters using ioctl(2).
171 cstream -t 176400 -i /dev/dsp0 -I f -o /dev/dsp1 -O f
173 Similar soundcard emulator, except that it allows you to grab the
174 data your applications sends to it from the other fifo, while
175 still having precise timing.
176 cstream -Oa -o /dev/dsp0 myhost.mydomain.com:17324
178 Connects port 3333 on host myhost.mydomain.com and whatever data
179 it finds there will be sent to the soundcard, with appropriate
180 settings for CD quality stero play.
181 cstream -i myaudiofile.raw -o :17324
183 This will open a TCP server on port 17324 and waits until someone
184 connects (for example, the commandline from the previous exam‐
185 ple). Then it will send the contents of myaudiofile.raw down the
186 TCP stream (for the previous audio example, typically a CD audio‐
187 track like you get from the tosha or cdparanoia utilities).
188 cstream -OD -o myfile
190 Write to file myfile with O_DIRECT. That usually means that the
192 filesystem buffer cache will not try to cache this file. You can
193 use that to prevent copying operations from eating up physical
194 memory. Note that when cstream encouters a write error it will
195 switch the output file from O_DIRECT to a normal file and write
196 all further blocks without O_DIRECT if writes without O_DIRECT
197 succeed. In practice that usually means that your last block, if
198 not a multiple of the filesystem block size, will still be writ‐
199 ten into the file (the maximum amount of data written without
200 O_DIRECT is your blocksize minus one). That way cstream ensures
201 that the output file has the length of the input, however odd the
202 length was and no matter what restrictions your OS places on
203 O_DIRECT output. Again, cstream will *not* pad the output to the
204 block size, you get the same file and file size as if not using
205 O_DIRECT, at the cost of switching to non-O_DIRECT whenever a
206 block is not the right size.
207 cstream -i :3333 | dd obs=8192 | ./cstream -omyfile -v7 -OD
209 This is what you need to do to buffer TCP input, so that the last
210 cstream will not switch away from O_DIRECT prematurely because of
211 short reads. If your input can do short reads (e.g. from TCP),
212 and you want to ensure that O_DIRECT stays in effect, you need a
213 buffer between the TCP stream and the O_DIRECT stream. Since
214 cstream does not yet support different input and output block
215 sizes, dd is suitable here. Note that this is only neccessary if
216 the OS requires multiples of the filesystem block size for O_DI‐
217 RECT. At the time of this writing this construct is needed on
218 Linux for using TCP streams with O_DIRECT, but it is not needed
219 on FreeBSD.
220 cstream -OS -o myfile
222 Writes to file myfile with O_SYNC. This means by the time the
223 system call returns the data is known to be on disk. This is not
224 the same thing as O_DIRECT. O_DIRECT can do its own buffering,
225 with O_SYNC there is no buffering at all. At the time of this
226 writing, O_SYNC on both Linux and FreeBSD is very slow (1/5th to
227 1/10th of normal write) and O_DIRECT is reasonably fast (1/4th to
228 1/2 of normal write). You can combined O_SYNC and O_DIRECT.
229
231 Exit code 0 means success.
232 Exit code 1 means a commandline syntax usage error.
234 Exit code 2 means other errors, especially system errors.
236
238 There should be an option to begin writing directly after the first read
239 ended and then fill the buffer with reads in the background. Right now
240 writing will not begin before the reader has filled the buffer completely
241 for the first time.
242 Not a bug: the code to do O_DIRECT is reasonably sophisticated. It will
244 fall back to normal I/O on errors. But before doing that it knows about
245 both filesystem blocksize requirements (will default I/O blocksize to
246 whatever the filesystem of the output file is in) and page alignment re‐
247 quirements (I/O will happen from a page-aligned buffer). However, the
248 combination of concurrent read/writes (-c options) and O_DIRECT has not
249 been tested bejond basic verification that it gets some tests right.
250
252 dd(1), mkfifo(2)
253
255 cstream was initially written by Martin Cracauer in 1998. For updates
256 and more information see http://www.cons.org/cracauer/cstream.html
257BSD March, 30, 1999 BSD