1SGP_DD(8) SG3_UTILS SGP_DD(8)
2
6 sgp_dd - copy data to and from files and devices, especially SCSI de‐
7 vices
8
10 sgp_dd [bs=BS] [count=COUNT] [ibs=BS] [if=IFILE] [iflag=FLAGS] [obs=BS]
11 [of=OFILE] [oflag=FLAGS] [seek=SEEK] [skip=SKIP] [--help] [--version]
12 [bpt=BPT] [coe=0|1] [cdbsz=6|10|12|16] [deb=VERB] [dio=0|1] [sync=0|1]
14 [thr=THR] [time=0|1] [verbose=VERB] [--dry-run] [--verbose]
15
17 Copy data to and from any files. Specialised for "files" that are Linux
18 SCSI generic (sg) and raw devices. Similar syntax and semantics to
19 dd(1) but does not perform any conversions. Uses POSIX threads (often
20 called "pthreads") to increase the amount of parallelism. This improves
21 speed in some cases.
22 The first group in the synopsis above are "standard" Unix dd(1) oper‐
24 ands. The second group are extra options added by this utility. Both
25 groups are defined below.
26
28 bpt=BPT
29 each IO transaction will be made using BPT blocks (or less if
30 near the end of the copy). Default is 128 for block sizes less
31 that 2048 bytes, otherwise the default is 32. So for bs=512 the
32 reads and writes will each convey 64 KiB of data by default
33 (less if near the end of the transfer or memory restrictions).
34 When cd/dvd drives are accessed, the block size is typically
35 2048 bytes and bpt defaults to 32 which again implies 64 KiB
36 transfers.
37 bs=BS where BS must be the block size of the physical device. Note
39 that this differs from dd(1) which permits 'bs' to be an inte‐
40 gral multiple of the actual device block size. Default is 512
41 which is usually correct for disks but incorrect for cdroms
42 (which normally have 2048 byte blocks).
43 cdbsz=6 | 10 | 12 | 16
45 size of SCSI READ and/or WRITE commands issued on sg device
46 names. Default is 10 byte SCSI command blocks (unless calcula‐
47 tions indicate that a 4 byte block number may be exceeded, in
48 which case it defaults to 16 byte SCSI commands).
49 coe=0 | 1
51 set to 1 for continue on error. Only applies to errors on sg de‐
52 vices. Thus errors on other files will stop sgp_dd. Default is
53 0 which implies stop on any error. See the 'coe' flag for more
54 information.
55 count=COUNT
57 copy COUNT blocks from IFILE to OFILE. Default is the minimum
58 (of IFILE and OFILE) number of blocks that sg devices report
59 from SCSI READ CAPACITY commands or that block devices (or their
60 partitions) report. Normal files are not probed for their size.
61 If skip=SKIP or seek=SEEK are given and the count is deduced
62 (i.e. not explicitly given) then that count is scaled back so
63 that the copy will not overrun the device. If the file name is a
64 block device partition and COUNT is not given then the size of
65 the partition rather than the size of the whole device is used.
66 If COUNT is not given and cannot be deduced then an error mes‐
67 sage is issued and no copy takes place.
68 deb=VERB
70 outputs debug information. If VERB is 0 (default) then there is
71 minimal debug information and as VERB increases so does the
72 amount of debug (max debug output when VERB is 9).
73 dio=0 | 1
75 default is 0 which selects indirect IO. Value of 1 attempts di‐
76 rect IO which, if not available, falls back to indirect IO and
77 notes this at completion. If direct IO is selected and
78 /proc/scsi/sg/allow_dio has the value of 0 then a warning is is‐
79 sued (and indirect IO is performed) For finer grain control use
80 'iflag=dio' or 'oflag=dio'.
81 ibs=BS if given must be the same as BS given to 'bs=' option.
83 if=IFILE
85 read from IFILE instead of stdin. If IFILE is '-' then stdin is
86 read. Starts reading at the beginning of IFILE unless SKIP is
87 given.
88 iflag=FLAGS
90 where FLAGS is a comma separated list of one or more flags out‐
91 lined below. These flags are associated with IFILE and are ig‐
92 nored when IFILE is stdin.
93 obs=BS if given must be the same as BS given to 'bs=' option.
95 of=OFILE
97 write to OFILE instead of stdout. If OFILE is '-' then writes to
98 stdout. If OFILE is /dev/null then no actual writes are per‐
99 formed. If OFILE is '.' (period) then it is treated the same
100 way as /dev/null (this is a shorthand notation). If OFILE exists
101 then it is _not_ truncated; it is overwritten from the start of
102 OFILE unless 'oflag=append' or SEEK is given.
103 oflag=FLAGS
105 where FLAGS is a comma separated list of one or more flags out‐
106 lined below. These flags are associated with OFILE and are ig‐
107 nored when OFILE is /dev/null, '.' (period), or stdout.
108 seek=SEEK
110 start writing SEEK bs-sized blocks from the start of OFILE. De‐
111 fault is block 0 (i.e. start of file).
112 skip=SKIP
114 start reading SKIP bs-sized blocks from the start of IFILE. De‐
115 fault is block 0 (i.e. start of file).
116 sync=0 | 1
118 when 1, does SYNCHRONIZE CACHE command on OFILE at the end of
119 the transfer. Only active when OFILE is a sg device file name.
120 thr=THR
122 where THR is the number or worker threads (default 4) that at‐
123 tempt to copy in parallel. Minimum is 1 and maximum is 1024.
124 time=0 | 1
126 when 1, the transfer is timed and throughput calculation is per‐
127 formed, outputting the results (to stderr) at completion. When 0
128 (default) no timing is performed.
129 verbose=VERB
131 increase verbosity. Same as deb=VERB. Added for compatibility
132 with sg_dd and sgm_dd.
133 -d, --dry-run
135 does all the command line parsing and preparation but bypasses
136 the actual copy or read. That preparation may include opening
137 IFILE or OFILE to determine their lengths. This option may be
138 useful for testing the syntax of complex command line invoca‐
139 tions in advance of executing them.
140 -h, --help
142 outputs usage message and exits.
143 -v, --verbose
145 when used once, this is equivalent to verbose=1. When used twice
146 (e.g. "-vv") this is equivalent to verbose=2, etc.
147 -V, --version
149 outputs version number information and exits.
150
152 Here is a list of flags and their meanings:
153 append causes the O_APPEND flag to be added to the open of OFILE. For
155 normal files this will lead to data appended to the end of any
156 existing data. Cannot be used together with the seek=SEEK op‐
157 tion as they conflict. The default action of this utility is to
158 overwrite any existing data from the beginning of the file or,
159 if SEEK is given, starting at block SEEK. Note that attempting
160 to 'append' to a device file (e.g. a disk) will usually be ig‐
161 nored or may cause an error to be reported.
162 coe continue on error. When given with 'iflag=', an error that is
164 detected in a single SCSI command (typically 'bpt' blocks) is
165 noted (by an error message sent to stderr), then zeros are sub‐
166 stituted into the buffer for the corresponding write operation
167 and the copy continues. Note that the sg_dd utility is more so‐
168 phisticated in such error situations when 'iflag=coe'. When
169 given with 'oflag=', any error reported by a SCSI WRITE command
170 is reported to stderr and the copy continues (as if nothing went
171 wrong).
172 dio request the sg device node associated with this flag does direct
174 IO. If direct IO is not available, falls back to indirect IO
175 and notes this at completion. If direct IO is selected and
176 /proc/scsi/sg/allow_dio has the value of 0 then a warning is is‐
177 sued (and indirect IO is performed).
178 direct causes the O_DIRECT flag to be added to the open of IFILE and/or
180 OFILE. This flag requires some memory alignment on IO. Hence
181 user memory buffers are aligned to the page size. Has no effect
182 on sg, normal or raw files.
183 dpo set the DPO bit (disable page out) in SCSI READ and WRITE com‐
185 mands. Not supported for 6 byte cdb variants of READ and WRITE.
186 Indicates that data is unlikely to be required to stay in device
187 (e.g. disk) cache. May speed media copy and/or cause a media
188 copy to have less impact on other device users.
189 dsync causes the O_SYNC flag to be added to the open of IFILE and/or
191 OFILE. The 'd' is prepended to lower confusion with the
192 'sync=0|1' option which has another action (i.e. a synchronisa‐
193 tion to media at the end of the transfer).
194 excl causes the O_EXCL flag to be added to the open of IFILE and/or
196 OFILE.
197 mmap can only be used in the iflag=FLAGS or the oflag=FLAGS argument
199 list but not both. The nominated side of the copy will use mem‐
200 ory mapped IO based on the mmap(2) system call. The sg driver
201 will remap its DMA destination or source buffer into the user
202 space when the mmap(2) system call is used on a sg device.
203 fua causes the FUA (force unit access) bit to be set in SCSI READ
205 and/or WRITE commands. This only has effect with sg devices. The
206 6 byte variants of the SCSI READ and WRITE commands do not sup‐
207 port the FUA bit. Only active for sg device file names.
208 null has no affect, just a placeholder.
210
212 Here are some retired options that are still present:
213 coe=0 | 1
215 continue on error is 0 (off) by default. When it is 1, it is
216 equivalent to 'iflag=coe oflag=coe' described in the FLAGS sec‐
217 tion above. Similar to 'conv=noerror,sync' in dd(1) utility.
218 Default is 0 which implies stop on error. More advanced coe=1
219 processing on reads is performed by the sg_dd utility.
220 fua=0 | 1 | 2 | 3
223 force unit access bit. When 3, fua is set on both IFILE and
224 OFILE; when 2, fua is set on IFILE;, when 1, fua is set on
225 OFILE; when 0 (default), fua is cleared on both. See the 'fua'
226 flag.
227
229 A raw device must be bound to a block device prior to using sgp_dd.
230 See raw(8) for more information about binding raw devices. To be safe,
231 the sg device mapping to SCSI block devices should be checked with 'cat
232 /proc/scsi/scsi' before use.
233 Raw device partition information can often be found with fdisk(8) [the
235 "-ul" argument is useful in this respect].
236 Various numeric arguments (e.g. SKIP) may include multiplicative suf‐
238 fixes or be given in hexadecimal. See the "NUMERIC ARGUMENTS" section
239 in the sg3_utils(8) man page.
240 The COUNT, SKIP and SEEK arguments can take 64 bit values (i.e. very
242 big numbers). Other values are limited to what can fit in a signed 32
243 bit number.
244 Data usually gets to the user space in a 2 stage process: first the
246 SCSI adapter DMAs into kernel buffers and then the sg driver copies
247 this data into user memory (write operations reverse this sequence).
248 This is called "indirect IO" and there is a 'dio' option to select "di‐
249 rect IO" which will DMA directly into user memory. Due to some issues
250 "direct IO" is disabled in the sg driver and needs a configuration
251 change to activate it.
252 All informative, warning and error output is sent to stderr so that
254 dd's output file can be stdout and remain unpolluted. If no options are
255 given, then the usage message is output and nothing else happens.
256 Why use sgp_dd? Because in some cases it is twice as fast as dd (mainly
258 with sg devices, raw devices give some improvement). Another reason is
259 that big copies fill the block device caches which has a negative im‐
260 pact on other machine activity.
261
263 The signal handling has been borrowed from dd: SIGINT, SIGQUIT and SIG‐
264 PIPE output the number of remaining blocks to be transferred and the
265 records in + out counts; then they have their default action. SIGUSR1
266 causes the same information to be output yet the copy continues. All
267 output caused by signals is sent to stderr.
268
270 Looks quite similar in usage to dd:
271 sgp_dd if=/dev/sg0 of=t bs=512 count=1MB
273 This will copy 1 million 512 byte blocks from the device associated
275 with /dev/sg0 (which should have 512 byte blocks) to a file called t.
276 Assuming /dev/sda and /dev/sg0 are the same device then the above is
277 equivalent to:
278 dd if=/dev/sda of=t bs=512 count=1000000
280 although dd's speed may improve if bs was larger and count was corre‐
282 spondingly scaled. Using a raw device to do something similar on a ATA
283 disk:
284 raw /dev/raw/raw1 /dev/hda
286 sgp_dd if=/dev/raw/raw1 of=t bs=512 count=1MB
287 To copy a SCSI disk partition to an ATA disk partition:
289 raw /dev/raw/raw2 /dev/hda3
291 sgp_dd if=/dev/sg0 skip=10123456 of=/dev/raw/raw2 bs=512
292 This assumes a valid partition is found on the SCSI disk at the given
294 skip block address (past the 5 GB point of that disk) and that the par‐
295 tition goes to the end of the SCSI disk. An explicit count is probably
296 a safer option.
297 To do a fast copy from one SCSI disk to another one with similar geome‐
299 try (stepping over errors on the source disk):
300 sgp_dd if=/dev/sg0 of=/dev/sg1 bs=512 coe=1
302
304 The exit status of sgp_dd is 0 when it is successful. Otherwise see the
305 sg3_utils(8) man page. Since this utility works at a higher level than
306 individual commands, and there are 'coe' and 'retries' flags, individ‐
307 ual SCSI command failures do not necessary cause the process to exit.
308
310 Written by Douglas Gilbert and Peter Allworth.
311
313 Report bugs to <dgilbert at interlog dot com>.
314
316 Copyright © 2000-2020 Douglas Gilbert
317 This software is distributed under the GPL version 2. There is NO war‐
318 ranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PUR‐
319 POSE.
320
322 A simpler, non-threaded version of this utility but with more advanced
323 "continue on error" logic is called sg_dd and is also found in the
324 sg3_utils package. The lmbench package contains lmdd which is also in‐
325 teresting. raw(8), dd(1)
326sg3_utils-1.45 February 2020 SGP_DD(8)