1DDPT(8)                              DDPT                              DDPT(8)
2
3
4

NAME

6       ddpt  -  copies data between files and storage devices. Support for de‐
7       vices that understand the SCSI command set.
8

SYNOPSIS

10       ddpt [bpt=BPT[,OBPC]] [bs=BS]  [cdbsz=IO_CDBSZ]  [cdl=CDL]  [coe={0|1}]
11       [coe_limit=CL] [conv=CONVS] [count=COUNT] [ddpt=VERS] [delay=MS[,W_MS]]
12       [ibs=IBS]   [id_usage=LIU]   if=IFILE    [iflag=FLAGS]    [intio={0|1}]
13       [iseek=SKIP]  [ito=ITO] [list_id=LID] [obs=OBS] [of=OFILE] [of2=OFILE2]
14       [oflag=FLAGS]   [oseek=SEEK]   [prio=PRIO]   [protect=RDP[,WRP]]   [re‐
15       tries=RETR]   [rtf=RTF]  [rtype=RTYPE]  [seek=SEEK]  [skip=SKIP]  [sta‐
16       tus=STAT]  [to=TO]  [verbose=VERB]  [--dry-run]  [--flexible]  [--help]
17       [--job=JF]  [--odx]  [--prefetch]  [--progress]  [--quiet]  [--verbose]
18       [--verify] [--version] [--wscan] [--xcopy] [ddpt] [JF]
19
20       For comparison here is the synopsis for GNU's dd command:
21
22       dd [bs=BS] [cbs=CBS] [conv=CONVS]  [count=COUNT]  [ibs=IBS]  [if=IFILE]
23       [iflag=FLAGS]    [obs=OBS]    [of=OFILE]    [oflag=FLAGS]   [seek=SEEK]
24       [skip=SKIP] [status=STAT] [--help] [--version]
25

DESCRIPTION

27       Copies data between files or simply reads data from  a  file.  Alterna‐
28       tively  if  the  --verify option is given, the IFILE and OFILE contents
29       are compared, stopping if an inequality is found. This utility is  spe‐
30       cialized  for  "files"  that are storage devices, especially those that
31       can use the SCSI command sets (e.g. SATA and SAS disks). It  can  issue
32       SCSI commands in pass-through ("pt") mode. Similar syntax and semantics
33       to the Unix dd(1) command.
34
35       For comparison, the SYNOPSIS section above shows both the ddpt  command
36       line operands and options followed by GNU's dd(1) command line operands
37       and options. Broadly speaking ddpt can be considered a super-set of dd.
38       See  the  section on DD DIFFERENCES for significant differences between
39       ddpt and dd.
40
41       This utility either does direct copies, based on read-write  sequences,
42       or  offloaded  copies.  In an offloaded copy the data being copied does
43       not necessarily pass through the memory of the the machine  originating
44       the  copy  operation;  this  can  save a significant amount of time and
45       lessen CPU usage.
46
47       When doing a direct copy, this utility breaks the  copy  into  segments
48       since  computer  RAM  is typically a scarce resource. First it reads in
49       BPT*IBS bytes from IFILE (or less if near the end of the copy)  into  a
50       copy  buffer.  In the absence of the various operand and flags that by‐
51       pass the write operation, the copy buffer is then written out to OFILE.
52       The  copy process continues working its way along IFILE and OFILE until
53       either COUNT is exhausted, an end of file is detected, or an error  oc‐
54       curs.  If  IBS  and  OBS are different, ddpt restricts the value of OBS
55       such that the copy buffer is an integral number of output blocks  (i.e.
56       (((IBS  * BPT) % OBS) == 0) ). In the following descriptions, "segment"
57       refers to all or part of a copy buffer.
58
59       The term "pt device" is used for a pass-through device  to  which  SCSI
60       commands  like  READ(10), WRITE(10) or POPULATE TOKEN may be sent. A pt
61       device may only be able to process SCSI commands in which case the "pt"
62       flag  is  assumed.  The  ability to recognize such a pt only device may
63       vary depending on the operating system  (e.g.  in  Linux  /dev/sg2  and
64       /dev/bsg/3:0:1:0  are  recognized). However if a device can process ei‐
65       ther normal UNIX read()/ write() calls or  pass-through  SCSI  commands
66       then  the default is to use UNIX read()/write() calls. That default can
67       be overridden by using the "pt"  flag  (e.g.  "if=/dev/sdc  iflag=pt").
68       When  pt  access is specified any partition information is ignored.  So
69       "if=/dev/sdc2 iflag=pt skip=3" will start at logical block address 3 of
70       '/dev/sdc'.  As  a protection measure ddpt will only accept that if the
71       force flag is also given (i.e. 'iflag=pt,force').
72
73       This utility supports two types of offloaded copies. Both are based  on
74       the  EXTENDED  COPY (XCOPY or xcopy) family of SCSI commands. The first
75       uses the XCOPY(LID1) command to do a disk to disk copy. LID1 stands for
76       List  IDentifier length of 1 byte and the commands are described in the
77       SPC-4 and earlier SPC-3 and SPC-2 standards. The SPC-4  standard  (ANSI
78       INCITS  513-2015)  added  the  XCOPY(LID4) sub-family of copy offloaded
79       commands. Now SPC-5 drafts have dropped the LID1 variants  and  removed
80       the LID4 suffix on the remaining XCOPY family of commands. To differen‐
81       tiate, this man page will continue to use the LID1 and  LID4  suffixes.
82       There  is  a  subset  of XCOPY(LID4), specialized for offloaded disk to
83       disk copies, that is known by the market name: ODX. In the descriptions
84       below  "xcopy" refers to copies based on XCOPY(LID1) while "odx" refers
85       to either full or partial ODX copies.  See the XCOPY and  ODX  sections
86       below for more information.
87
88       The  syntax  of the dd command is somewhat unique in Unix and ddpt fol‐
89       lows in a similar fashion. Operands (i.e. those with the  <name>=<some‐
90       thing> structure) are shown in OPERANDS section. The more familiar Unix
91       options (i.e. those starting with one or two hyphens) are shown in  the
92       OPTIONS  section. Then there are a few arguments which are command line
93       entities that are neither operands nor options, see the ARGUMENTS  sec‐
94       tion.
95

OPERANDS

97       The operands are listed alphabetically (by <name>) below. The <name> is
98       the part that is to the left of the equal sign. All <names> start  with
99       a lower case alphabetical character.
100
101       bpt=BPT[,OBPC]
102              where  BPT  is  Blocks Per Transfer. A direct copy is made up of
103              multiple transfers, each first reading BPT  input  blocks  (i.e.
104              BPT  *  IBS bytes) from IFILE into the copy buffer and then from
105              that copy buffer writing (BPT * IBS)  /  OBS  output  blocks  to
106              OFILE.  This continues until the copy is finished, with the last
107              transfer being potentially shorter. The default BPT value varies
108              depending  on IBS. When IBS < 8, BPT is 8192; when IBS < 64, BPT
109              is 1024; when IBS < 1024, BPT is 128; when IBS <  8192,  BPT  is
110              16;  when  IBS < 32768, BPT is 4; else BPT defaults to 1. If BPT
111              is given as 0 it is treated as the default value.  For "bs=512",
112              BPT  defaults to 128 so that 64 KiB (or less) is read from IFILE
113              into the copy buffer. This operand is treated differently in ODX
114              and is typically only needed for testing; see ODX section.
115              The  optional  OBPC  (Output Blocks Per Check) argument controls
116              the granularity of sparse writes, write sparing and trim checks.
117              The default granularity is the size of the copy buffer (i.e. BPT
118              * IBS bytes). That can be reduced by specifying OBPC. The finest
119              granularity  is  when  OBPC  is 1 which implies the unit of each
120              check is OBS bytes.  When OBPC is 0, or not given,  the  default
121              granularity  is used. Large OBPC values are rounded down so that
122              OBPC*OBS does not exceed the size of the copy buffer.
123              odx: may be used to limit the  data  represented  by  each  ROD.
124              Mainly for testing.
125              If BPT is too large on Linux, the obscure "Invalid argument" er‐
126              ror value (EINVAL) is returned.
127
128       bs=BS  where BS is the IFILE and OFILE block size in bytes.   Conflicts
129              with  either  the  "ibs=" or "obs=" operands. The value of BS is
130              placed in IBS and OBS.  If IFILE or OFILE is a "pt" device  then
131              BS must be the logical block size of the device. See the DD DIF‐
132              FERENCES section below. The default is 512 bytes unless overrid‐
133              den  by  the  DDPT_DEF_BS  environment variable. Note that newer
134              disks use 4096 byte blocks with perhaps larger block sizes  com‐
135              ing  in  the future. CD/DVD/BD media use a logical block size of
136              2048 bytes.
137
138       cdbsz=IO_CDBSZ
139              size of SCSI  READ  and/or  WRITE  (VERIFY)  command  descriptor
140              blocks (cdb) in bytes. IO_CDBSZ may be one number or two numbers
141              separated by a comma.  The acceptable numbers are 0, 6, 10,  12,
142              16 or 32. The default 0 will usually be set to 10 internally un‐
143              less the (first and last) LBAs cannot fit in 32  bits  in  which
144              case  the 16 byte variant of each command is used. If one number
145              is given it applies both to the IFILE and IFILE. If two  numbers
146              are  given,  the  first applies to the IFILE (i.e. the READ com‐
147              mand) and the second applies to the OFILE.
148              If IFILE or OFILE is not a SCSI  pass-through  device  then  the
149              corresponding IO_CDBSZ value is ignored.
150
151       cdl=CDL
152              allows  setting of command duration limits. CDL is either a sin‐
153              gle value or two values separated by a comma. If  one  value  is
154              given,  it  applies  to  both  IFILE  and  OFILE  (if  they  are
155              pass-through devices). If two values are given,  the  first  ap‐
156              plies  to IFILE while the second applies to OFILE. The value may
157              be from 0 to 7 where 0 is the default and  means  there  are  no
158              command  duration  limits. Command duration limits are only sup‐
159              ported by 16 and 32 byte READ and WRITE commands (and the  WRITE
160              SCATTERED  command  which  is  not used by this utility). If the
161              cdbsz operand is not given and would have a value less than  16,
162              then if CDL is greater than 0, the cdbsz is increased to 16.
163              Command  duration limits can be accesses and changed in the Com‐
164              mand duration limit A and B mode pages, plus the  Command  dura‐
165              tion  limit  T2A  and  T2B mode pages. The sdparm utility may be
166              used to access and change these mode pages.
167
168       coe={0|1}
169              set to 1 for continue on error. Applies to errors on  input  and
170              output  for  pt  devices but only on input from block devices or
171              regular files. Errors on other files will stop ddpt. Default  is
172              0  which  implies stop on any error. See the 'coe' flag for more
173              information.
174
175       coe_limit=CL
176              where CL is the maximum number of consecutive bad blocks stepped
177              over due to "coe=1" on reads before the copy terminates. The de‐
178              fault is 0 which is implies no limit. This operand is  meant  to
179              stop  the  copy  soon  after  unrecorded media is detected while
180              still offering "continue on error"  capability  for  infrequent,
181              randomly distributed errors.
182
183       conv=CONVS
184              see the CONVERSIONS section below.
185
186       count=COUNT
187              copy  COUNT input blocks from IFILE to OFILE. If this operand is
188              not given (or COUNT is '-1') then the COUNT may be deduced  from
189              either IFILE or OFILE. See the COUNT section below.
190              If  a 'hard' gather list is given to skip=SKIP or a 'hard' scat‐
191              ter list is given to seek=SEEK then typically count=COUNT should
192              not  be  supplied.  This is because a 'hard' scatter gather list
193              implies a transfer count. If both are given then ddpt will  exit
194              if they are unequal, the force flag can be used to override this
195              action. See the SCATTER GATHER LISTS section below for a discus‐
196              sion of 'hard' and 'soft' scatter gather lists.
197
198       ddpt=VERS
199              causes a syntax error while parsing the command line if the cur‐
200              rent version of the ddpt utility is less  than  VERS.  VERS  can
201              take  one  of  two forms: starting with a digit in which case is
202              should have the form "<major_vn>.<minor_vn>"  or  starting  with
203              the  letter  "r" followed by "<svn.rev>". The latter case is the
204              subversion revision number. Both numbers can  be  found  in  the
205              output  of  the --version option. The purpose of this operand is
206              to be placed in job files so that they are not run on older ver‐
207              sions of this utility
208
209       delay=MS[,W_MS]
210              after each segment is copied (typically every (IBS * BPT) bytes)
211              a delay (sleep) of MS milliseconds  is  performed.  The  default
212              value  for  MS is 0 which implies no delay. If W_MS is given and
213              greater than 0 (its default value) then there is  an  additional
214              delay of W_MS milliseconds associated with each actual write op‐
215              eration that is performed.  If MS is greater than 0  then  there
216              is  not  a  delay  before  the  first copy segment (or after the
217              last); if W_MS is greater than 0 then there is not a  delay  be‐
218              fore  the  first  write  segment. These delays can be used for a
219              bandwidth limiting.
220              odx: the MS delay is implemented in the same fashion after  each
221              ROD  is  copied,  apart from the last. If W_MS is greater than 0
222              then that delay occurs before each WUT command, apart  from  the
223              first.
224
225       ibs=IBS
226              where IBS is the IFILE block size in bytes. The default value is
227              BS or its default (512). Conflicts the "bs=" operand (i.e.  giv‐
228              ing both "bs=512 ibs=512" is considered a syntax error).
229
230       id_usage=LIU
231              xcopy:  SCSI EXTENDED COPY parameter list LIST ID USAGE field is
232              set to LIU. The default value is 0 or 2 . LIU can  be  a  number
233              between  0  and  3 inclusive or a string. The strings can be ei‐
234              ther: 'hold' for 0, 'discard' for 2 or 'disable' for 3.
235
236       if=IFILE
237              read from IFILE. If IFILE is '-'  then  stdin  is  read.  Starts
238              reading at the beginning of IFILE unless SKIP is given.
239              This  operand  must  be  given (apart from one odx case and when
240              iflag=00 or iflag=ff is given).
241              odx: the rtf=RTF operand may replace the if=IFILE operand as in‐
242              put. See the ODX section.
243
244       iflag=FLAGS
245              where  FLAGS is a comma separated list of one or more flags out‐
246              lined in the FLAGS section below.  These  flags  are  associated
247              with IFILE and are mostly ignored when IFILE is stdin.
248
249       intio={0|1}
250              set to 1 for allow signals (SIGINT, SIGPIPE and SIGUSR1 (or SIG‐
251              INFO)) to be received during IO from IFILE or  IO  to  OFILE  or
252              OFILE2.   Default  is  0 which causes these signals to be masked
253              during IO operations with a check for signals prior each IO.  As
254              long  as  IO  operations don't lock up (e.g. SCSI READ and WRITE
255              commands) the default is the safer option. Even if IO operations
256              do lock up it is best to let the kernel take care of that.
257
258       iseek=SKIP
259              in its simplest form, SKIP is a single number: start reading af‐
260              ter SKIP blocks (each of IBS bytes) from  the  start  of  IFILE.
261              Default  is block 0 (i.e. start of file). This operand is a syn‐
262              onym for skip=SKIP, see its description.
263
264       ito=ITO
265              odx: ITO is the inactivity timeout whose units are seconds.  The
266              default  value  is  0 which means the copy manager will take the
267              default inactivity timeout value from the Block Device ROD Token
268              Limits  descriptor  in the Third Party Copy VPD page. ITO is ig‐
269              nored if it it exceeds the maximum inactivity timeout  value  in
270              the same descriptor (unless the force flag is given).
271
272       list_id=LID
273              LID  is  the xcopy LIST IDENTIFIER field or the STR_ID field for
274              the WRITE STREAM command. Fo xcopy it is used  to  associate  an
275              originating  xcopy  command  with follow-up commands such as RE‐
276              CEIVE ROD TOKEN INFORMATION. If given, the LID should not  clash
277              with any other xcopy LID currently in use on this I_T nexus.
278              xcopy:  LID  is  a 1 byte (8 bit) value whose default value is 1
279              or, if id_usage=disable, 0 . LID must not exceed 255.
280              odx: LID is a 4 byte (32 bit) value whose default value  is  257
281              (i.e.  0x101) and, if a second default is needed, 258 (0x102) is
282              used. If a clash is detected  on  the  default  list  identifier
283              value then the next higher value is tried (stopping after 10 at‐
284              tempts).
285              oflag=wstream: LID is a 2 byte  (16  bit)  value  whose  default
286              value  is  0. It is the Stream Identifier (STR_ID field) for the
287              WRITE STREAM(16) command. Valid Stream identifiers  are  0x1  to
288              0xffff (65535) inclusive, so the default value of 0 is invalid.
289
290       obs=OBS
291              where OBS is the OFILE block size in bytes. The default value is
292              BS or its default (512). Conflicts the "bs=" operand (e.g.  giv‐
293              ing both "bs=512 obs=512" is considered a syntax error).  If OBS
294              is given then it has the following restriction: the integer  ex‐
295              pression  (((IBS  *  BPT) % OBS) == 0) must be true.  Stated an‐
296              other way: the copy buffer size must be an integral multiple  of
297              OBS. If of2=OFILE2 is given then OBS is its block size as well.
298
299       of=OFILE
300              write to OFILE. The default value is /dev/null . If OFILE is '-'
301              then writes to stdout. If OFILE  is  /dev/null  then  no  actual
302              writes  are  performed.  If  OFILE  is  '.'  (period) then it is
303              treated the same way as /dev/null . If OFILE exists then  it  is
304              _not_ truncated unless "oflag=trunc" is given. See section on DD
305              DIFFERENCES.
306              odx: if this operand (of=OFILE) is not given and the rtf=RTF op‐
307              erand  is given then the RTF file may be thought of as receiving
308              the output in the form of one or more ROD Tokens.  See  the  ODX
309              section.
310
311       of2=OFILE2
312              write output to OFILE2. The default action is not to do this ad‐
313              ditional write (i.e. when this operand is not given). OFILE2  is
314              assumed  to  be  a  regular  file or a fifo (i.e. a named pipe).
315              OFILE2 is opened for writing and is  created  if  necessary.  If
316              OFILE2  is a fifo (named pipe) then some other command should be
317              consuming that data (e.g. 'md5sum OFILE2'), otherwise this util‐
318              ity  will  block. The write to OFILE2 occurs before the write to
319              OFILE and prior to sparse writing and write  sparing  logic.  So
320              everything read is written to OFILE2.
321              OFILE2 is not truncated before writing. Assuming that the OFILE2
322              length is shorter than what is written (or it is  created)  then
323              its  contents  should be the concatenation of all segments (each
324              of ibs*bpt bytes long, with  the  last  segment  being  possibly
325              shorter).  The  gather  list  given to skip=SKIP effects what is
326              read into each segment so it indirectly effects what is  written
327              to  OFILE2.  However  the scatter list given to seek=SEEK has no
328              effect on what is written to OFILE2.
329
330       oflag=FLAGS
331              where FLAGS is a comma separated list of one or more flags  out‐
332              lined  in  the  FLAGS  section.  These flags are associated with
333              OFILE and are ignored when OFILE is /dev/null, '.' (period),  or
334              stdout.
335
336       oseek=SEEK
337              start  writing SEEK blocks (each of OBS bytes) from the start of
338              OFILE. Default is block 0 (i.e. start of file). This operand  is
339              a synonym for seek=SEEK, see its description.
340
341       prio=PRIO
342              xcopy:  SCSI  EXTENDED COPY parameter list PRIORITY field is set
343              to PRIO.  The default value is 1 .
344
345       protect=RDP[,WRP]
346              where RDP is the RDPROTECT field in SCSI READ commands  and  WRP
347              is the WRPROTECT field in SCSI WRITE commands. The default value
348              for both is 0 which implies no additional protection information
349              will  be  transferred.   Both RDP and WRP can be from 0 to 7. If
350              RDP is greater than 0 then IFILE must be a pt device. If WRP  is
351              greater than 0 then OFILE must be a pt device.
352              When  copying  data plus protection information from one disk to
353              another then 'protect=3,3' will give the least number  of  prob‐
354              lems  as  that  combination then of PI checking on both the read
355              and write side. See the PROTECTIO INFORMATION section below.
356
357       retries=RETR
358              sometimes retries at the host are useful, for example when there
359              is  a  transport error. When RETR is greater than zero then SCSI
360              READs and WRITEs are retried on error, RETR times. Default value
361              is zero.  Only applies to errors on pt devices.
362
363       rtf=RTF
364              odx: where RTF is a filename. One or more ROD tokens are written
365              to RTF during a read to tokens variant or a full  copy  variant.
366              One or more ROD tokens are read from RTF during a write from to‐
367              ken variant.  This operand is not required on a full copy  vari‐
368              ant.  ROD Tokens are 512 bytes long and an extra 8 byte (big-en‐
369              dian) integer containing the 'number of  bytes  represented'  is
370              placed after each ROD Token if rtf_len is given.
371
372       rtype=RTYPE
373              odx:  where  RTYPE  is the ROD Type. The default value (0) indi‐
374              cates that the copy manager (in the source) decides.  RTYPE  can
375              be  a decimal number, a hex number (prefixed by 0x or with a "h"
376              appended)  or  one   of   "pit-def",   "pit-vuln",   "pit-pers",
377              "pit-cow", "pit-any" or "zero".  The final truncated word can be
378              spelt out (e.g.  "pit-vulnerable").   The  "pit-"  prefix  is  a
379              shortening  of "point in time" copy. The "zero" causes a special
380              Block device zero Token to be created.
381
382       seek=SEEK
383              start writing SEEK blocks (each of OBS bytes) from the start  of
384              OFILE.  Default  is block 0 (i.e. start of file). The SEEK value
385              may exceed the number of OBS-sized blocks in OFILE.
386              SEEK can be a scatter (gather) list:  see  the   SCATTER  GATHER
387              LISTS section below.
388
389       skip=SKIP
390              start  reading SKIP blocks (each of IBS bytes) from the start of
391              IFILE. Default is block 0 (i.e. start of file). The  SKIP  value
392              must be less than the number of IBS-sized blocks in IFILE.
393              SKIP  can  be  a  (scatter)  gather list: see the SCATTER GATHER
394              LISTS section below.
395
396       status=STAT
397              the STAT value of 'noxfer' suppresses the throughput  speed  and
398              the  copy time reporting at the end of the copy. A STAT value of
399              'none' additionally suppresses the records in and out  reporting
400              after  the  copy.  So 'status=none' makes ddpt act like a tradi‐
401              tional Unix command in which "no news is good  news".   The  de‐
402              fault action of ddpt is to show the throughput (in megabytes per
403              second) and the time taken to do the copy after the "records in"
404              and  "records out" lines at the end of the copy. A STAT value of
405              'sgl' together with '-vv' option will print internally generated
406              scatter  gather lists before the copy begins. When the '-vv' op‐
407              tions is given alone then internal  scatter  lists  headers  are
408              printed, but not individual elements.  In most cases these scat‐
409              ter gather lists will be the same lists given to  the  seek=SEEK
410              and  skip=SKIP  operands.   As a convenience the value 'null' is
411              accepted for STAT and does nothing.
412              A STAT value of 'progress' prints a progress report (to  stderr)
413              every  two  minutes.  If 'progress' is used twice, either by re‐
414              peating the  'status=progress'  operand  or  by  entering  'sta‐
415              tus=progress,progress',  then a progress report is printed every
416              minute. If it is used thrice, the a progress report  is  printed
417              every  30  seconds.  Note that care is taken not to flood the OS
418              with calls to check the time which  would  slow  down  the  copy
419              process.  The  amount of data output by the progress reports can
420              modified at runtime (e.g. during a long copy).  If  the  verbose
421              flag  is 0 or 1 (but not higher and not if the --quiet option is
422              given) then sending a SIG_USR1 (or SIGINFO in FreeBSD) signal to
423              the  running ddpt process will toggle the verbose flag between 0
424              and 1. Note there is now a shorter form, the  command  line  op‐
425              tion: --progress or simply '-p'.
426              Note that GNU's dd supports 'noxfer', 'none' and 'progress' with
427              similar semantics.
428
429       to=TO  odx, xcopy: where TO is am xcopy originating command timeout  in
430              seconds.   The  default value is 0 which is converted internally
431              to 600 seconds (10 minutes). Best to set this timeout value well
432              above  the  expected copy time.  In a odx full copy this timeout
433              is applied to both the POPULATE TOKEN and WRITE USING TOKEN com‐
434              mands.
435
436       verbose=VERB
437              as  VERB increases so does the amount of debug reporting sent to
438              stderr.  Default value is zero which yields the  minimum  amount
439              of debug reporting.  A value of 1 reports extra information that
440              is not repetitive. A value 2 reports cdbs and responses for SCSI
441              commands  that  are  not  repetitive  (i.e.  other that READ and
442              WRITE). Error processing is not considered repetitive. Values of
443              3  and 4 yield reporting for all SCSI commands, plus Unix read()
444              and write() calls, so there can be a lot of output.
445              If VERB is "-1" then reporting that  would  have  been  sent  to
446              stderr  is redirected to /dev/null essentially throwing it away.
447              It has the same action as the --quiet option.
448              In some cases the position of verbose=VERB (or --verbose) on the
449              command  line  is significant. For example to debug (or at least
450              list out) a scatter gather list given to skip=SKIP or  seek=SEEK
451              the  verbose=VERB  operand  (or  --verbose) should appear before
452              skip and/or seek.
453

OPTIONS

455       Options are listed in alphabetical order, sorted by their long name.
456
457       -d, --dry-run
458              does all the operand and option processing, opens given file and
459              devices  but  bypasses  the copy stage. For complex command line
460              invocations or for testing invocations to be  placed  in  script
461              files, this option may be useful to check for syntax and related
462              errors.
463              When used once the logic bypasses the copy just before it  would
464              normally start copying. When used twice (or more) it goes deeper
465              into the copy to the IO call level before bypassing  the  calls.
466              To  see (on stderr) information for each IO this option combina‐
467              tion may be useful: '-ddvv'.
468
469       -f, --flexible
470              this option currently only effects the parsing of sgl_s in files
471              that are in hexadecimal plus they have a leading line with 'HEX'
472              in them. Without this option any such line must be invoked  with
473              'H@'  before the filename; in other words the 'H' in the invoca‐
474              tion needs to match the HEX in the file. With this option a  sgl
475              in  a  file  can  be  invoked with '@' and if a line with HEX is
476              parsed before any LBA,NUM pairs then it switches to  hexadecimal
477              mode; so all the parsed LBA,NUM pairs are assumed to be in hexa‐
478              decimal.
479
480       -h, --help
481              reports usage message then exits.
482
483       --job=JF
484              where JF is a file name. That file can contain operands and  op‐
485              tions  listed  in  this  and  the previous sections. See the JOB
486              FILES section below.
487
488       -o, --odx
489              indicates to this utility that one of the four odx  variants  is
490              requested.  See ODX section.
491
492       -P, --prefetch
493              this  option  is  only  active  when the --verify option is also
494              given.  It causes the SCSI PRE-FETCH(OFILE, IMMED) command to be
495              sent  at  the start of each copy segment. See the VERIFY section
496              below.
497
498       -p, --progress
499              this option has  the  same  effect  as  status=progress  but  is
500              shorter to type and easier to remember. When given once, the de‐
501              fault reporting period is two  minutes,  if  given  twice  (e.g.
502              '-pp') that period is shortened to one minute.
503
504       -q, --quiet
505              redirects  the  messages  (sent to stderr) to /dev/null which is
506              essentially throwing them away. That redirect takes place  after
507              the  command line operands and options are parsed and associated
508              sanity checks performed.
509
510       -v, --verbose
511              equivalent of verbose=1. If --verbose appears twice then that is
512              equivalent to verbose=2. Also -vv is equivalent to verbose=2.
513
514       -X, --verify
515              rather  than copy, a comparison is done between IFILE and OFILE.
516              The compare continues until an  inequality  is  found  at  which
517              point  the operation stops. This is only available if OFILE is a
518              pass-through device that implements VERIFY(10 or 16) with BYTCHK
519              set to 1. See the VERIFY section below.
520
521       -V, --version
522              reports version number information then exits.
523
524       -w, --wscan
525              this  option  is available in Windows only. It lists storage de‐
526              vice names and the corresponding  volumes,  if  any.  When  used
527              twice  it  adds  the "bus type" of the closest transport (e.g. a
528              SATA disk in a USB connected enclosure has bus type  USB).  When
529              used  three  times  a SCSI adapter scan is added. When used four
530              times only a SCSI adapter scan is shown.  See  EXAMPLES  section
531              below and the README.win32 file.
532
533       -x, --xcopy
534              this  option  will  attempt to call the SCSI EXTENDED COPY(LID1)
535              command. In the absence of another indication the xcopy  command
536              will be sent to the destination (i.e. OFILE). See the section on
537              ENVIRONMENT VARIABLES below.
538

ARGUMENTS

540       Arguments do not start with hyphen nor contain a "=".
541
542       ddpt   this string is just a marker. It must not appear in the  command
543              line and it must appear in the contents of a job file that isn't
544              part of a comment (i.e. following a "#" on a line). A syntax er‐
545              ror  is  generated  (and no copy occurs) if these rules are vio‐
546              lated.
547
548       JF     a command line element that does not contain a '=' (i.e. a  ddpt
549              operand)  and  does  not  start  with '-', apart from the string
550              "ddpt" is treated as a job file (i.e. JF).  See  the  JOB  FILES
551              section below.
552

CONVERSIONS

554       One  or  more  conversions  can be given to the "conv=" option. If more
555       than one is given, they should be comma separated. ddpt does  not  per‐
556       form  the  traditional  dd conversions (e.g. ASCII to EBCDIC). Recently
557       added conversions inherited from GNU's dd  overlap  somewhat  with  the
558       some of ddpt flags.
559
560       fdatasync
561              equivalent  to  "oflag=fdatasync".  Flushes data associated with
562              the OFILE to storage at the end of the copy. This conversion  is
563              for compatibility with GNU's dd.
564
565       fsync  equivalent  to "oflag=fsync". Flushes data and meta-data associ‐
566              ated with the OFILE to storage at the end of the copy. This con‐
567              version
568
569       no_del_tkn
570              equivalent to "oflag=no_del_tkn".
571
572       nocreat
573              OFILE  will  not be created if it doesn't exist. The default ac‐
574              tion if OFILE does not exist is to create a regular  file  (then
575              write  into it). The default action can be surprising if writing
576              to a device node in /dev and due to some external action (e.g. a
577              USB key being removed) that device node name disappears.
578
579       noerror
580              this  conversion  is very close to "iflag=coe" and is treated as
581              such. See the "coe" flag. Note that an error on a  block  device
582              or regular file OFILE will stop the copy.
583
584       notrunc
585              this  conversion  is  accepted for compatibility with dd and ig‐
586              nored since the default action of this utility is not  to  trun‐
587              cate OFILE.
588
589       null   has no affect, just a placeholder.
590
591       prefer_rcs
592              equivalent to "oflag=prefer_rcs".
593
594       resume See "resume" in the FLAGS sections for more information.
595
596       rtf_len
597              equivalent to "oflag=rtf_len".
598
599       sparing
600              See "sparing" in the FLAGS sections for more information.
601
602       sparse FreeBSD's  dd  supports  "conv=sparse"  and now GNU's dd does as
603              well so the same syntax is supported in ddpt.  See  "sparse"  in
604              the FLAGS sections for more information.
605
606       sync   is  ignored  by  ddpt. With dd it means supply zero fill (rather
607              than skip) and is typically used like  this  "conv=noerror,sync"
608              to have the same functionality as ddpt's "iflag=coe".
609
610       trunc  if  OFILE  is  a regular file then truncate it prior to starting
611              the copy. See "trunc" in the FLAGS section.
612

FLAGS

614       A list of flags and their meanings follow. The flag name is followed by
615       one  or two indications in square brackets. The first indication is ei‐
616       ther "[i]", "[o]" or "[io]" indicating this  flag  is  active  for  the
617       IFILE,  OFILE  or  both  the IFILE and the OFILE. The second indication
618       contains some combination of "reg",  "blk"  "pt",  "odx",  or  "xcopy".
619       These  indicate whether the flag applies to a regular file, a block de‐
620       vice (accessed via Unix read() and write() commands, a pass-through de‐
621       vice,  an  ODX  offloaded  copy or a XCOPY(LID1) offloaded copy respec‐
622       tively.  Other special file types that are sometimes  referred  to  are
623       "fifo" and "tape".
624
625       00 [i] This  flag  may replace IFILE with a source of zero (0x0) bytes.
626              If IFILE is given and is shorter than OFILE then it continues to
627              copy  after  IFILE  is  exhausted supplying zero fill bytes. Can
628              only be used on input. Zeros can  also  be  generated  by  using
629              "if=/dev/zero" or an equivalent.
630              If  both '00' and 'ff' flags are given then a marching byte pat‐
631              tern is placed in the segment prior to writing it out. It starts
632              at  0x0 and wraps after 0xff (if the segment is large enough, as
633              it usually is).
634
635       append [o] [reg], [io] [odx]
636              causes the O_APPEND flag to be added to the open of  OFILE.  For
637              regular  files  this will lead to data being appended to the end
638              of any existing data. Conflicts the seek=SEEK  option.  The  de‐
639              fault  action  of this utility is to overwrite any existing data
640              from the beginning of OFILE or, if SEEK is  given,  starting  at
641              block  SEEK.  Note  that attempting to 'append' to a device file
642              (e.g. a disk) will usually be ignored or may cause an  error  to
643              be reported.
644              odx:  if  the  rtf=RTF option is given, RTF exists, is a regular
645              file and this utility wants to write to RTF then new ROD  Tokens
646              are  appended  to RTF. The default action is to truncate RTF be‐
647              fore new ROD Tokens are written to it.
648
649       atomic [o] [pt]
650              this flag changes the pass-through SCSI  WRITE  command  to  the
651              SCSI    WRITE    ATOMIC(16)    command   on   OFILE   (and   the
652              cdbsz={6|10|12|16|32} option is ignored for OFILE). If this flag
653              is applied to IFILE or to a non pass-through file then it is ig‐
654              nored.
655
656       block [io] [pt]
657              pass-through file opens are non-blocking by default and may  re‐
658              port  the  pt  device is busy. Use this flag to open blocking so
659              utility may wait until another process locking (or with  an  ex‐
660              clusive open) is complete before continuing.
661
662       bytchk [o] [pt]
663              only  active  when  used  together  with oflag=wverify. Sets the
664              BYTCHK field in the SCSI WRITE AND VERIFY  command.  Since  that
665              field  is  two  bits  wide,  this flag can be specified multiple
666              times (up to three) to place  the  corresponding  value  in  the
667              field.
668
669       cat [io] [xcopy]
670              xcopy:   set  CAT  (residual  data  handling)  bit  in  EXTENDED
671              COPY(LID1) parameter list segment descriptor header. May  appear
672              in either flag list when xcopy is being used. Works with the PAD
673              bit for handling residual data on the destination side. See  the
674              XCOPY section below.
675
676       coe [io] [pt], [i] [reg,blk]
677              continue on error. 'iflag=coe oflag=coe' and 'coe=1' are equiva‐
678              lent.  Errors occurring on output regular or  block  files  will
679              stop ddpt.  Error messages are sent to stderr. This flag is sim‐
680              ilar to 'conv=noerror,sync' in the  dd(1)  utility.  Unrecovered
681              errors are counted and reported in the summary at the end of the
682              copy.
683
684              This paragraph concerns coe on pt devices. A medium, hardware or
685              blank  check  error  during a read operation will will cause the
686              following: first re-read blocks prior to the bad block, then try
687              to  recover  the  bad block (supplying zeros if that fails), and
688              finally re-read the blocks after the bad block. A medium,  hard‐
689              ware  or  blank check error while writing is reported but other‐
690              wise ignored. SCSI disks may automatically try and remap  faulty
691              sectors  (see the AWRE and ARRE in the read write error recovery
692              mode page (the sdparm utility can access these attributes)).  If
693              bad  LBAs  are  reported by the pass-through then the LBA of the
694              lowest and highest bad block is also reported.
695
696              This paragraph concerns coe on input regular files and block de‐
697              vices.   When  a  EIO or EREMOTEIO error is detected on a normal
698              segment read then the segment is re-read  one  block  (i.e.  IBS
699              bytes) at a time. Any block that yields a EIO or EREMOTEIO error
700              is replaced by zeros. Any other error, a short read or an end of
701              file  will  terminate  the copy, usually after the data that has
702              been read is written to the output file.
703
704       dc [io] [blk,pt]
705              xcopy: set DC (destination counter) bit in  EXTENDED  COPY(LID1)
706              parameter  list  segment descriptor header. May appear in either
707              flag list when xcopy is being used.
708
709       direct [io] [reg,blk]
710              causes the O_DIRECT flag to be added to the open of IFILE and/or
711              OFILE.  This  flag  requires  some memory alignment on IO. Hence
712              user memory buffers are aligned to the page size.  May  have  no
713              effect  on  pt  devices  or  cause an error (e.g. Linux seems to
714              dis-allow O_DIRECT on character devices (like sg devices) yield‐
715              ing  EINVAL).  This  flag will bypass caching/buffering normally
716              done by block layer. Beware of data coherency issues if the same
717              locations have been recently accessed via the block layer in its
718              normal mode (i.e. non-direct). See open(2) man page.
719
720       dpo [io] [pt]
721              set the DPO bit (disable page out) in SCSI READ and  WRITE  com‐
722              mands.  Not supported for 6 byte cdb variants of READ and WRITE.
723              Indicates that data is unlikely to be required to stay in device
724              (e.g.  disk)  cache.   May speed media copy and/or cause a media
725              copy to have less impact on other device users.
726
727       errblk [i] [pt] [experimental]
728              attempts to create or append to a file  called  "errblk.txt"  in
729              the current directory the logical block addresses of blocks that
730              cannot be read. The first (appended) line  is  "#  start  <time‐
731              stamp>".  That is followed by the LBAs in hex (and prefixed with
732              "0x") of any block that cannot be read, one LBA per line. If the
733              sense  data does not correctly identify the LBA of the first er‐
734              ror in the range it was asked to read then a LBA  range  is  re‐
735              ported  in  the  form  of  the lowest and the highest LBA in the
736              range separated by a "-". At the end of the copy a line with  "#
737              stop  <timestamp>"  is  appended to "errblk.txt". Typically used
738              with "coe".
739
740       excl [io] [reg,blk]
741              causes the O_EXCL flag to be added to the open of  IFILE  and/or
742              OFILE. See open(2) man page.
743
744       fdatasync [o] [reg,blk]
745              Flushes  data associated with the OFILE to storage at the end of
746              the copy.
747
748       ff [i] This flag may replace IFILE with a  source  of  0xff  bytes.  If
749              IFILE  is  given  and is shorter than OFILE then it continues to
750              copy after IFILE is exhausted supplying  0xff  fill  bytes.  Can
751              only be used on input.
752              If  both '00' and 'ff' flags are given then a marching byte pat‐
753              tern is placed in the segment prior to writing it out. It starts
754              at  0x0 and wraps after 0xff (if the segment is large enough, as
755              it usually is).
756
757       flock [io] [reg,blk,pt]
758              after opening the associated file (i.e. IFILE and/or  OFILE)  an
759              attempt  is  made  to  get  an  advisory exclusive lock with the
760              flock()  system  call.  The  flock  arguments  are  "FLOCK_EX  |
761              FLOCK_NB"  which  will  cause  the lock to be taken if available
762              else a "temporarily unavailable" error  is  generated.  An  exit
763              status of 90 is produced in the latter case and no copy is done.
764              See flock(2) man page.
765
766       force [io] [pt] [xcopy,odx]
767              override difference between given block size and the block  size
768              found  by  the  SCSI  READ CAPACITY command. Use the given block
769              size. Without this flag the copy would not be performed. pt  ac‐
770              cess  to what appears to be a block partition is aborted in ver‐
771              sion 0.92; that can be overridden by the force flag. For related
772              reasons  the  'norcap' flag requires this flag when applied to a
773              block device accessed via pt.
774              xcopy and odx: various limits imposed by associated VPD pages or
775              the  RECEIVE COPY OPERATING PARAMETERS command can be overridden
776              (i.e.  exceeded) if this flag is given. Note that the copy  man‐
777              ager will probably object.
778
779       fsync [o] [reg,blk]
780              Flushes  data and metadata (describing the file) associated with
781              the OFILE to storage at the end of the copy.
782
783       fua [io] [pt]
784              causes the FUA (force unit access) bit to be set  in  SCSI  READ
785              and/or  WRITE commands. The 6 byte variants of the SCSI READ and
786              WRITE commands do not support the FUA bit.
787
788       fua_nv [io] [pt]
789              causes the FUA_NV (force unit access non-volatile cache) bit  to
790              be  set in SCSI READ and/or WRITE commands. This only has an ef‐
791              fect with pt devices.  The 6 byte variants of the SCSI READ  and
792              WRITE commands do not support the FUA_NV bit. The FUA_NV bit was
793              made obsolete in SBC-3 revision 35d.
794
795       ignoreew [o] [tape]
796              ignore the early warning indication (of end of tape) when  writ‐
797              ing to tape.  See TAPE section.
798
799       immed [io] [odx]
800              sets the IMMED bit in the POPULATE TOKEN (when [i]) or WRITE US‐
801              ING TOKEN (when [o]) command. That command should return  status
802              promptly after starting the data transfer. The RECEIVE ROD TOKEN
803              INFORMATION command is then used to poll  for  completion.  SCSI
804              command  timeouts  should  not  be exceeded, even for very large
805              RODs, if this flag is used.
806
807       nocache [io] [reg,blk]
808              use posix_fadvise(POSIX_FADV_DONTNEED) to  advise  corresponding
809              file there is no need to fill the file buffer with recently read
810              or written blocks. If used with "iflag=" it  will  increase  the
811              read ahead on IFILE.
812
813       no_del_tkn [o] [odx]
814              will clear the DEL_TKN bit on the last WRITE USING TOKEN command
815              of each ROD Token in a odx full copy. In a large odx  full  copy
816              several  ROD  Tokens  may be used (one after the other). The de‐
817              fault action is to set the DEL_TKN bit on the last  WUT  command
818              of  each  ROD. Either way it should not make much difference be‐
819              cause the copy manager deletes a ROD Token when  its  inactivity
820              time-out occurs.
821
822       nocreat [o]
823              if  OFILE  does not exist then an error will be generated rather
824              than creating an empty regular file. This flag has the same  ac‐
825              tion as "conv=nocreat".
826
827       nofm [o] [tape]
828              no  File  Mark (FM) on close when writing to tape. See TAPE sec‐
829              tion.
830
831       nopad [o] [tape]
832              when the block to be written to a tape drive contains less  than
833              OBS bytes, then this option causes the partial block to be writ‐
834              ten as is. The default action for a tape in this case is to  pad
835              the block. See TAPE section.
836
837       norcap [io] [pt]
838              do  not  perform SCSI READ CAPACITY command on the corresponding
839              pt device.  If used on block device accessed via pt then 'force'
840              flag  is also required. This is to warn about using pt access on
841              what may be a block device partition.
842
843       nowrite [o] [reg,blk,pt]
844              bypass writes to OFILE. The "records out" count  is  not  incre‐
845              mented.  OFILE is still opened but "oflag=trunc" if given is ig‐
846              nored. Also the ftruncate call associated with the  sparse  flag
847              is ignored (i.e.  bypassed). Commands such as trim and SCSI SYN‐
848              CHRONIZE CACHE are still sent.
849
850       null [io]
851              has no affect, just a placeholder.
852
853       odx [io] [odx]
854              indicates to this utility that one of the four  variants  of  an
855              odx  copy  is  requested.  Using  any  of  the --odx, rtf=RTF or
856              rtype=RTYPE options also indicates that odx  is  requested.  See
857              the ODX section.
858
859       pad [o] [reg,blk,pt], [io] [xcopy]
860              when the block to be written (typically the last block) contains
861              less than OBS bytes, then this option causes  the  block  to  be
862              padded  with  zeros (i.e. bytes of binary zero). The default ac‐
863              tion for a regular file and a fifo is to do a partial write. The
864              default  action of a block and a pt device is to ignore the par‐
865              tial write. The default action of a tape is to pad, so this flag
866              is not needed (see the nopad flag).
867              xcopy: sets the PAD bit in the CSCD descriptor of the associated
868              IFILE or OFILE. Is associated with residual  data  handling  and
869              works together with the cat flag. See the XCOPY section below.
870
871       prealloc [o] [reg]
872              use  the  fallocate() call prior to starting a copy to set OFILE
873              to its expected size.
874
875       prefer_rcs [o] [odx]
876              prefer RECEIVE COPY STATUS (RCS) command to the RECEIVE ROD  TO‐
877              KEN  INFORMATION  (RRTI) command which is the default. This only
878              is active when polling after a WUT command (since polling  after
879              a  PT  command needs to fetch the ROD Token so it needs the RRTI
880              command).
881
882       pt [io] [blk,pt]
883              causes a device to be accessed in "pt" mode. In "pt"  mode  SCSI
884              READ  and  WRITE  commands are sent to access blocks rather than
885              standard UNIX read() and write() commands. The "pt" mode may  be
886              implicit  if  the device is only capable of passing through SCSI
887              commands (e.g. the  /dev/sg*  and  some  /dev/bsg/*  devices  in
888              Linux).  This  flag  is  needed for device nodes that can be ac‐
889              cessed both via standard UNIX read()  and  write()  commands  as
890              well as SCSI commands. Such devices default standard UNIX read()
891              and write() commands in the absence of this flag.
892
893       rarc [i] [pt]
894              bit set in READ(10, 12, 16 and  32)  to  suppress  RAID  rebuild
895              functions  when a bad (or recovered after difficulties) block is
896              detected.
897
898       resume [o] [reg]
899              when a copy is interrupted (e.g. with Control-C  from  the  key‐
900              board) then using the same invocation again with the addition of
901              "oflag=resume" will attempt to restart the copy from  the  point
902              of  the interrupt (or just before that point). It is harmless to
903              use "oflag=resume" when OFILE doesn't exist or is  zero  length.
904              If  the  length  of OFILE is greater than or equal to the length
905              implied by a ddpt invocation that includes  "oflag=resume"  then
906              no further data is copied.
907
908       self [io] [pt]
909              used together with trim flag to do a self trim (trim of segments
910              of a pt device that contain all zeros). If OFILE is  not  given,
911              then it is set to the same as IFILE. If SEEK is not given it set
912              to the same value as SKIP (possibly adjusted if IBS and OBS  are
913              different). Implicitly sets "nowrite" flag.
914
915       sparing [o] [reg,blk,pt]
916              during  the  copy each IBS * BPT byte segment is read from IFILE
917              into a buffer. Then, instead of writing that  buffer  to  OFILE,
918              the  corresponding  segment is read from OFILE into another buf‐
919              fer. If the two buffers are  different,  the  former  buffer  is
920              written  to the OFILE. If the two buffers compare equal then the
921              write to OFILE is not performed. Write sparing is useful when  a
922              write  operation is significantly slower than a read. Under some
923              conditions flash memory devices have slow writes plus  an  upper
924              limit on the number of times the same cell can be rewritten. The
925              granularity of the comparison can be reduced  from  the  default
926              IBS  *  BPT  byte  segment  with the the OBPC value given to the
927              "bpt=" option. The finest granularity is when OBPC  is  1  which
928              implies OBS bytes.
929
930       sparse [io] [reg,blk,pt]
931              after  each  IBS  *  BPT  byte segment is read from IFILE, it is
932              checked to see if it is all zeros. If so, that  segment  is  not
933              written to OFILE. See the section on SPARSE WRITES below for the
934              difference between using this flag once or twice. The  granular‐
935              ity of the zero comparison can be reduced from the default IBS *
936              BPT byte segment with the OBPC value given to the "bpt=" option.
937              The sparse flag may be used on input when a file is  only  being
938              read (e.g.  when of=OFILE is not given or OFILE is /dev/null) to
939              determine how many blocks are contained in  sparse  segments  of
940              IFILE.
941
942       ssync [o] [pt]
943              if OFILE is in "pt" mode then the SCSI SYNCHRONIZE CACHE command
944              is sent to OFILE at the end of the copy.
945
946       strunc [o] [reg]
947              perform a sparse copy with a ftruncate system call to extend the
948              length of the OFILE if required. Sets the sparse flag internally
949              if this has not been specified on  the  command  line.  See  the
950              sparse flag and the section on SPARSE WRITES below.
951
952       sync [io] [reg,blk]
953              causes  the  O_SYNC flag to be added to the open of IFILE and/or
954              OFILE. See open(2) man page.
955
956       rtf_len [io] [odx]
957              odx: with the 'read to tokens' variant, after 512 bytes of  each
958              ROD  Token  are written to RTF an additional 8 byte (big endian)
959              integer is written. That integer is the number of bytes that the
960              associated  ROD represents. The draft standards say for standard
961              ROD types the ROD Token contains this value. However vendor spe‐
962              cific ROD types may be used or vendors may choose not to comply.
963              Either way the 'write from tokens' variant  needs  to  know  the
964              data size associated with the ROD it is writing from.
965
966       trim [io] [pt] [experimental]
967              similar  logic  to the "sparse" option. However instead of skip‐
968              ping segments that are full of zeros a "trim" command is sent to
969              OFILE. Usually set as an oflag argument but for self trim can be
970              used as an iflag argument (e.g. "iflag=self,trim"). Depending on
971              the  usage this may require the device to support "deterministic
972              read zero after trim". See the TRIM, UNMAP AND WRITE  SAME  sec‐
973              tion below.
974
975       trunc [o] [reg]
976              if  OFILE is a regular file then it is truncated prior to start‐
977              ing the copy. If SEEK is not given or 0 then OFILE is  truncated
978              to  zero  length;  when  SEEK is larger than zero the truncation
979              takes  place  at  file  byte  pointer  SEEK*OBS.    Ignored   if
980              "oflag=append". Conflicts with "oflag=sparing".
981
982       unmap [io] [pt]
983              same as the trim flag.
984
985       wverify [o] [pt]
986              this  causes  SCSI WRITE AND VERIFY commands to be sent to OFILE
987              (instead of SCSI WRITE (or WRITE ATOMIC)  commands).  Note  that
988              the  fua  flag  is  ignored  when this flag is given. The BYTCHK
989              field in the SCSI WRITE AND VERIFY commands is set to  zero  un‐
990              less the bytchk flag is also given.
991
992       wstream [o] [pt]
993              this  causes  SCSI  WRITE STREAM(16) command to be sent to OFILE
994              (instead of SCSI WRITE. The Stream Identify (valid range:  1  to
995              0xffff  (65535))  should  be  given  via the list_id=LID operand
996              (note that it defaults to 0 which is invalid). The stream can be
997              created  (closed  (for  write)  or  its status checked) with the
998              sg_stream_ctl utility. It is the user's responsibility to open a
999              stream before calling "ddpt ... oflag=wstream list_id=<strm_id>"
1000              and close it after, if required.
1001
1002       xcopy [io] [pt]
1003              invoke SCSI XCOPY(LID1) logic and send the XCOPY command to  the
1004              either  IFILE  or  OFILE depending on which flag this called. If
1005              both  are  given  (i.e.  an  invocation  including  'iflag=xcopy
1006              oflag=xcopy') then send the XCOPY(LID1) to OFILE.
1007

COUNT

1009       When the count=COUNT option is not given (or COUNT is '-1') then an at‐
1010       tempt is made to deduce COUNT as follows.
1011
1012       When both or either IFILE and OFILE are block devices, then the minimum
1013       size,  expressed in units of input blocks, is used. When both or either
1014       IFILE and OFILE are pass-through devices, then the  minimum  size,  ex‐
1015       pressed in units of input blocks, is used.
1016
1017       If a regular file is used as input, its size, expressed in units of in‐
1018       put blocks (and rounded up if necessary) is used. Note that the  round‐
1019       ing  up  of  the deduced COUNT may result in a partial read of the last
1020       input block and a corresponding partial write to OFILE if it is a regu‐
1021       lar file. After a regular file to regular file copy the length of OFILE
1022       will be the same as IFILE unless OFILE existed and its length  was  al‐
1023       ready  greater than that of IFILE. To get a copy like the standard Unix
1024       cp command, use oflag=trunc with ddpt.
1025
1026       The size of pt devices is deduced from the SCSI READ CAPACITY  command.
1027       Block device sizes (or their partition sizes) are obtained from the op‐
1028       erating system, if available.
1029
1030       If skip=SKIP or seek=SEEK are given and the COUNT is deduced (i.e.  not
1031       explicitly  given)  then that size is scaled back so that the copy will
1032       not overrun the file or device.
1033
1034       If COUNT is not given and IFILE is a fifo (and stdin is  treated  as  a
1035       fifo)  then  IFILE  is  read until an EOF is detected.  If COUNT is not
1036       given and IFILE is a /dev/zero (or equivalent) then zeros are read  un‐
1037       til an error occurs (e.g. file system full).
1038
1039       If  COUNT  is  not given and cannot be deduced then an error message is
1040       issued and no copy takes place.
1041

JOB FILES

1043       Some operands can have long arguments (e.g. skip=SKIP and  iflag=FLAGS)
1044       so  that  the  command  line can become quite long. Also scatter gather
1045       lists can be arbitrarily long and may be generated by a  program;  then
1046       it  would  be  tiresome  and error-prone to re-type them on the command
1047       line. So the job file was introduced to hold  this  utility's  operands
1048       and options.
1049
1050       A  job  file is invoked by either the --job=JF option or by placing the
1051       job filename (JF) unadorned on the command line. The job filename  can‐
1052       not  contain  a  "=",  start  with a hyphen nor be called "ddpt". It is
1053       parsed when it is detected, in a left to  right  scan  of  the  command
1054       line.  The  JF file must contain the string "ddpt" and may invoke other
1055       job files (to a maximum depth of 4). A job file should not  invoke  it‐
1056       self.  Also the first line of the job file should not contain any char‐
1057       acters (bytes) with their top bit set; in other words it should be  re‐
1058       stricted  to  7  bit ASCII (otherwise sanity checks might think it is a
1059       binary file and reject it).
1060
1061       The operands and options within a job file are processed in  the  order
1062       they are found (i.e. parsing lines left to right, top (of file) to bot‐
1063       tom).  The operands and options may contradict (and cause a syntax  er‐
1064       ror), override or accumulate with earlier ones, the same as if they ap‐
1065       peared on the command line. For example '-v' on the command  line  fol‐
1066       lowed  by  a job file containing '-vv' will result in a verbosity level
1067       of '-vvv' during the copy phase. Empty  lines,  lines  only  containing
1068       whitespace(s)  and anything from and including a '#' in a job file line
1069       are ignored.
1070

SCATTER GATHER LISTS

1072       Each element of a scatter gather list (sgl, plural: sgl_s) is  made  up
1073       of  a  starting logical block address (LBA, plural: LBAs), and a number
1074       of blocks (NUM) to be accessed from that starting LBA.
1075
1076       The skip=SKIP and seek=SEEK options (and their aliases) can take  scat‐
1077       ter  gather  lists.  These  can be explicit on the command line, fed in
1078       through stdin or in a file whose name is prefixed by "@" or "H@" on the
1079       command line. For large scatter gather lists, placing them in a file is
1080       the most practical as command lines  are  limited  in  length.  Scatter
1081       gather  list  (sgl) is a collective term for either a scatter list or a
1082       gather list.  The actual implementation of each sgl is an  array.  Syn‐
1083       tactically a scatter list and a gather list are the same.
1084
1085       Conceptually  these  sgl_s refer to what happens at the "far end" (e.g.
1086       within a hard disk or SSD), not what happens in the computer's  memory.
1087       So  a  gather list is associated with the read part of a copy (i.e. the
1088       first half) where a list of Logical Blocks (LBs,  identified  by  their
1089       addresses,  hence  LBAs)  and a number of consecutive, following blocks
1090       are "gathered" from the medium (e.g. a SSD). They  are  formed  into  a
1091       linear sequence of bytes that is transferred into a segment in the com‐
1092       puter's RAM. The second half of the copy, the write  part,  may  use  a
1093       scatter  list.  A  scatter list starts with a linear sequence of bytes,
1094       taken from the segment, that is transferred  to  the  device  and  then
1095       "scattered" on the medium as indicated by the list of LBA,NUM pairs.
1096
1097       In  the  simplest  case  a sgl is given on the command line and has the
1098       form: LBA1,NUM1[,LBA2,NUM2[,LBA3,NUM3...]]. There must be an even  num‐
1099       ber  of  items  (i.e.  for every LBAn there should be a following NUMn)
1100       with one exception: when LBA1 alone is given, in which case the value 0
1101       is  assumed  for  NUM1. Comma is the simplest separator for the command
1102       line, but whitespace may also be used (but needs to be escaped  because
1103       the  shell  usually interprets whitespace as an option separator). In a
1104       file (or read from stdin or file redirection) more flexibility is  per‐
1105       mitted  in  the format.  The LBA,NUM pairs could all appear on one line
1106       in a file but the line length is limited to  1024  characters  (with  a
1107       maximum of 256 parseable items on it). So for longer sgl_s one pair per
1108       line is recommended in file format.  Also  in  file  format  everything
1109       from  and including '#' to the end of that line is ignored as are lines
1110       that are empty or only contain whitespace(s).
1111
1112       Each pair becomes one element (or more, see below) of the sgl.  By  de‐
1113       fault  all  numbers given for LBA and NUM items are in decimal with op‐
1114       tional suffix multipliers. Hex numbers use either a "0x"  prefix  or  a
1115       'h'  suffix  (hex  notation and suffix multipliers cannot be mixed). In
1116       the case of a 'H@' lead-in to the filename on  the  command  line,  all
1117       numbers  are  interpreted  as hex with no suffix multipliers permitted.
1118       Further, with the 'H@' lead-in the file may contain  the  string  'HEX'
1119       before  any  numbers are given. The 'HEX' is ignored. The point of this
1120       is to catch when a sgl file with default hexadecimal numbers  is  given
1121       without  the  'H@'  lead-in; in this case this utility will exit saying
1122       that file is in the wrong format.  This "wrong format"  action  can  be
1123       bypassed with the --flexible option.
1124
1125       Allowing sgl_s brings lots of flexibility (including the possibility to
1126       use the SCSI WRITE SCATTERED command) but with that  comes  complexity.
1127       Every sgl is scanned to determine if it is monotonic and whether it has
1128       overlapping elements. The term monotonic is used  to  indicate  whether
1129       each LBA is in ascending order, with each LBA greater than the previous
1130       element's LBA.  Overlapping refers to the situation when any  element's
1131       LBA range intersects with any other element's range. Elements that have
1132       zero number of blocks (described here as "degenerate") are ignored  for
1133       determining monotonic and overlapping (and the lowest LBA). Overlapping
1134       elements are not ideal (but not necessarily fatal). The above mentioned
1135       WRITE  SCATTERED command allows the medium's logic to write elements in
1136       any order it prefers.  That means if elements overlap,  then  the  user
1137       doesn't  know  which one gets written last (overwriting the one written
1138       to the same LBA earlier).  Determining whether element  ranges  overlap
1139       is  difficult  in  the general case (so this utility doesn't do it) but
1140       easy in the case of a monotonic sgl (so this utility does do it). Warn‐
1141       ings  are  issued in dangerous situations, with the force flag allowing
1142       the warning to be overridden.
1143
1144       A degenerate sgl element is one that has zero in its NUM  field.   Nor‐
1145       mally degenerate elements are ignored with some exceptions. The defini‐
1146       tion of the SCSI WRITE SCATTERED command clearly states that degenerate
1147       elements are valid, thus do not cause an error, but cause no associated
1148       action. This utility uses the concept of a 'hard'  and  'soft'  sgl:  a
1149       'soft'  sgl  is  one in which the last element's NUM is zero (i.e.  its
1150       last element is degenerate). A sgl with a non-zero NUM in its last ele‐
1151       ment  is considered 'hard'. In a 'soft' sgl the LBA of the last element
1152       should be greater than or equal to any  LBA+NUM  of  earlier  elements.
1153       Because  this  is  hard to check it is not enforced, so the decision is
1154       made on whether a sgl is hard or soft simply by  checking  the  NUM  of
1155       that  last  element.  The difference between a hard and soft sgl is the
1156       way the sum of NUM of all elements is used  by  this  utility.   For  a
1157       'hard'  sgl  that  sum is used for COUNT when the count=COUNT option is
1158       not given; and if count=COUNT is given and the counts differ then those
1159       two values are output and this utility exits with a syntax error. For a
1160       'soft' sgl the degenerate last element  is  interpreted  as  "from  the
1161       highest  LBA in the list to the end of the copy" where the COUNT is de‐
1162       termined some other way. The "highest LBA" is calculated from all  ele‐
1163       ments  that  have  a non-zero number of blocks plus the LBA of the last
1164       element (regardless of whether it is degenerate or not).
1165
1166       The rules in the above paragraph make a one item skip or seek  argument
1167       (e.g.  skip=0x123) in this utility first become a one element sgl (e.g.
1168       containing the pair [0x123, 0x0]). Since this is the last  element,  it
1169       is  a  soft  sgl  and  the transfer will start from the given lba (i.e.
1170       0x123) and continues for the number of blocks indicated by  some  other
1171       mechanism (e.g.  an option such as count=COUNT or the length of IFILE).
1172       This mirrors what the classic dd command does with its skip= and  seek=
1173       options.
1174
1175       Some  sgl  implementation  details:  LBAs are stored in 64 bit integers
1176       which is more than sufficient to span even the largest disk  array  be‐
1177       hind a logical device, even if the block size is one byte, which is un‐
1178       likely. The NUM field is a 32 bit integer and this is more problematic.
1179       The  reason is that SCSI WRITE commands (and their variants) only allo‐
1180       cate at most a 32 bit integer for this value. Further, modern operating
1181       systems do not allow any driver to get large amounts of contiguous sys‐
1182       tem RAM, even if the machine has it available. A 32 bit integer for NUM
1183       with  each  block  at  512 bytes is around 2 TB of storage. Unix system
1184       calls (in Linux) also limit each read(2) and write(2) system call to 32
1185       bits  of  single  bytes  which is 4 GB. The problem for this utility is
1186       that the NUM can easily exceed 32 bits when  a  single  scatter  gather
1187       list element refers to the whole device. The action taken by this util‐
1188       ity is to allow larger than 32 bit NUM values to be given on  the  com‐
1189       mand line (or in a scatter gather list file). However such a large ele‐
1190       ment will be split into multiple elements internally. This will be vis‐
1191       ible  to the user when the verbose=VERB option (or one of its variants)
1192       is used with an elevated value.
1193
1194       There is a helper utility called ddpt_sgl in this package for  generat‐
1195       ing, manipulating and checking scatter gather lists. See its manpage.
1196

SANITY CHECKS

1198       With  powerful  data  tools,  the ability to accidentally overwrite and
1199       hence lose important data is ever present. So a significant portion  of
1200       the  code is dedicated to checking the input arguments for duplications
1201       and contradictions. Still nothing is better than re-reading the command
1202       line (which can be quite long) before hitting the enter key.
1203
1204       Other  useful  possibilities  are to use job files (see the JF argument
1205       and the --job=JF option) and the --dry-run option.  The "dry  run"  op‐
1206       tion  is  becoming popular in modern command line utilities and more or
1207       less does what the user would expect. Firstly it parses all the command
1208       line  arguments  then  opens IFILE, OFILE and OFILE2 as directed by the
1209       command line and does any meta-data operations that it would  typically
1210       do  (e.g.  check a pass-though or block device's logical block size and
1211       object if it differs from BS, IBS or  OBS  (whichever  applies)).  Then
1212       just  at  the  point  where the code would commence the actual copy (or
1213       read) it does a premature exit. If the --dry-run option is given twice,
1214       the  code continues into the copy logic and bypasses the low level read
1215       and write calls (and file repositioning). That inner level of "dry run"
1216       is  useful for debugging and can be used with multiple verbose=VERB op‐
1217       tions.
1218
1219       The verbose=VERB option sends diagnostic messages to stderr. The higher
1220       value  of VERB (in verbose=VERB) or the more times that -v is used, the
1221       greater the volume of diagnostic messages. When use three or more times
1222       then  diagnostic  messages  are  generated  for each read to, and write
1223       from, the working copy buffer; so the volume  of  messages  is  propor‐
1224       tional to the number of reads and writes that are done; this can easily
1225       be in the megabyte range. If used less than three times, the reads  and
1226       writes  associated  with  the  copy do not generate diagnostic messages
1227       (unless abnormal situations are encountered). These diagnostic messages
1228       are  mainly associated with command line parsing and fetching meta-data
1229       about the given files, plus messages from the cleanup at the end of the
1230       copy.
1231
1232       The following command line arguments are checked that they don't appear
1233       more than once: bpt=BPT[,OBPC], bs=BS, count=COUNT, ibs=IBS,  if=IFILE,
1234       iseek=SKIP,  obs=OBS,  of=OFILE,  of2=OFILE2, oseek=SEEK, seek=SEEK and
1235       skip=SKIP. On the other hand, some arguments are additive, for  example
1236       iflag=FLAGS,  oflag=FLAGS,  status=STAT and --verbose and may appear as
1237       many times as required.
1238

XCOPY

1240       This section describes XCOPY(LID1) support with this utility.  For  ODX
1241       support (XCOPY(LID4) subset) see the ODX section.
1242
1243       A  device (logical unit (LU)) that supports XCOPY operations should set
1244       the 3PC field (3PC stands for Third Party Copy) in its standard INQUIRY
1245       response. That is not checked when this utility does an xcopy operation
1246       but if it fails, that is one thing that the user may want to check.
1247
1248       If the xcopy starts and fails while underway, then 'sg_copy_results -s'
1249       may  be  useful  to  view the copy status. It might also be used from a
1250       different process with the same I_T nexus (i.e. the  same  machine)  to
1251       check status during an xcopy operation.
1252
1253       The  pad  and  cat  flags control the handling of residual data. As the
1254       data can be specified either in terms of source or  target  block  size
1255       and  both  might  have different block sizes residual data is likely to
1256       happen in these cases.  If both block sizes are  identical  these  bits
1257       have no effect as residual data will not occur.
1258
1259       If  neither  of  these flags are set, the EXTENDED COPY command will be
1260       aborted with additional sense 'UNEXPECTED INEXACT SEGMENT'.
1261
1262       If only the cat flag is set the residual data will be retained and made
1263       available  for  subsequent  segment  descriptors. Residual data will be
1264       discarded for the last segment descriptor.
1265
1266       If the pad flag is set for the source  descriptor  only,  any  residual
1267       data for both source or destination will be discarded.
1268
1269       If  the  pad  flag  is  set for the target descriptor only any residual
1270       source data will be handled as if the cat flag is set, but any residual
1271       destination data will be padded to make a whole block transfer.
1272
1273       If  the  pad flag is set for both source and target any residual source
1274       data will be discarded, and  any  residual  destination  data  will  be
1275       padded.
1276
1277       There   is   a   web   page   discussing   ddpt,   XCOPY   and  ODX  at
1278       https://sg.danny.cz/sg/ddpt_xcopy_odx.html
1279

ODX

1281       This section describes ODX support (an  XCOPY(LID4)  subset)  for  this
1282       utility.   ODX  descriptions  use  the following command name abbrevia‐
1283       tions: PT for the POPULATE TOKEN command, RRTI for the READ  ROD  TOKEN
1284       INFORMATION command, and WUT for the WRITE USING TOKEN command.
1285
1286       A  device  (logical unit (LU)) that supports ODX operations is required
1287       to set the 3PC field (3PC stands for Third Party Copy) in its  standard
1288       INQUIRY  response  and  support  the Third Party Copy VPD page. If this
1289       utility generates errors noting the absence of these then the device in
1290       question probably does not support ODX.
1291
1292       There a four variants of ODX supported by ddpt:
1293         full copy : ddpt --odx if=/dev/sg3 bs=512 of=/dev/sg4
1294         zero output blocks : ddpt if=/dev/null rtype=zero bs=512 of=/dev/sg4
1295         read to tokens : ddpt if=/dev/sg3 bs=512 skip=@gath.lst rtf=a.rt
1296         write from tokens : ddpt rtf=a.rt bs=512 of=/dev/sg4 seek=@scat.lst
1297
1298       The  full  copy will call PT and WUT commands repeatedly until the copy
1299       is complete. More precisely the full copy will make the largest  single
1300       call  to  PT  allowed by the input's Third Party Copy VPD page (and, if
1301       given, allowed by the BPT argument in the bpt=BPT[,OBPC] option).  Then
1302       one or more WUT calls are made to write out from the ROD created by the
1303       PT step. The largest single WUT call is  constrained  by  the  output's
1304       Third  Party Copy VPD page (and, if given, allowed by the OBPC argument
1305       in the bpt=BPT[,OBPC] option). This sequence continues  until  the  re‐
1306       quested copy is complete.
1307
1308       The  zero  output  blocks variant is a special case of the full copy in
1309       which only WUT calls are made. ODX defines a special ROD Token to  zero
1310       blocks. That special ROD Token has a fixed pattern (shown in SBC-3) and
1311       does not need to be created by a PT command like normal ROD Tokens.
1312
1313       The read to tokens and the write from tokens variants are  designed  to
1314       be  the read (input) and write (output) sides respectively of a network
1315       copy.  Each can run on different machines by sending the RTF file  from
1316       the  machine doing the read to the machine doing the write. The read to
1317       tokens will make one or more PT calls and output the resulting ROD  To‐
1318       kens to the RTF file. RTF might be a regular file or a named pipe.
1319
1320       All  four  variants can have the immed flag set. Then the PT and/or WUT
1321       commands are issued with the IMMED bit set and the RRTI command is used
1322       to  poll for completion. The delay between the polls is as suggested by
1323       the RRTI command (or if no suggestion is made, 500  milliseconds).  Ei‐
1324       ther  iflag=immed, oflag=immed or both can be given but are only effec‐
1325       tive if the corresponding IFILE or OFILE sends a PT or WUT command.
1326
1327       Typically there is no need to give the list_id=LID option. If this  op‐
1328       tion  is  not  given  then  257  is chosen. If that is busy then 258 is
1329       tried.  That continues until a usable LID is found or 10 LIDs have been
1330       tried.  In  the  latter case ddpt exits with status of 55 (operation in
1331       progress). If the user gives list_id=LID option and LID  is  busy  then
1332       ddpt exits with exit status 55.
1333
1334       If  the  block  size of the input and output are different (i.e. IBS is
1335       not equal to OBS) then one must be a multiple of the other. So an input
1336       block size of 512 bytes and an output block size of 4096 bytes (or vice
1337       versa) is acceptable.
1338
1339       The four ODX variants are distinguished  as  follows:  if  OFILE  is  a
1340       pass-through  device,  if=/dev/null (or equivalent) and rtype=zero then
1341       the zero output blocks variant is selected. If both IFILE and OFILE are
1342       pass-through  devices  and  there  is some indication of an ODX request
1343       (e.g.  the --odx option), then the full copy variant is  selected.  The
1344       read  to  tokens and the write from token variants are indicated by the
1345       absence of either a of=OFILE or a if=IFILE option,  respectively,  plus
1346       the presence of a rtf=RTF option.
1347
1348       The helper utility ddptctl contains options to issue a single PT, RRTI,
1349       WUT or COPY OPERATION ABORT command. It can  also  issue  a  series  of
1350       polling  RRTI  commands. It can decode information in ROD Tokens (which
1351       is not as informative as it should be) and print the number  of  blocks
1352       and block size of a disk, plus protection information if available. See
1353       ddptctl.
1354
1355       There  is  a   web   page   discussing   ddpt,   XCOPY   and   ODX   at
1356       https://sg.danny.cz/sg/ddpt_xcopy_odx.html
1357

SPARSE WRITES

1359       Bypassing  writes of blocks full of zeros can save a lot of IO. However
1360       with regular files, bypassed writes at the end of the copy can lead  to
1361       an  OFILE  which  is  shorter  than  it  would have been without sparse
1362       writes. This can lead to integrity checking programs  like  md5sum  and
1363       sha1sum generating different values.
1364
1365       This utility has two ways of handling this file length problem: writing
1366       the last block (even if it is full of zeros)  or  using  the  ftruncate
1367       system  call.  A  third approach is to ignore the problem (i.e. leaving
1368       OFILE shorter). The ftruncate  approach  is  used  when  "oflag=strunc"
1369       while the last block is written when "oflag=sparse". To ignore the file
1370       length issue use "oflag=sparse,sparse". Note that if OFILE's length  is
1371       already correct or longer than required, no action is taken.
1372
1373       The  support  for sparse writing of regular files may depend on the OS,
1374       the file system and the settings of OFILE. POSIX makes  few  guarantees
1375       when  the  ftruncate  system call is used to extend a file's length, as
1376       may occur when "oflag=strunc". Further,  primitive  file  systems  like
1377       VFAT  may  not  accept  sparse writes or simulate the effect by writing
1378       blocks of zeros. The latter approach will  defeat  any  sparse  writing
1379       performance gain.
1380

TRIM, UNMAP AND WRITE SAME

1382       This  is  a new storage feature often associated with Solid State Disks
1383       (SSDs) or disk arrays with "thin provisioning". In the ATA command  set
1384       (ACS-2)  the  relevant command is DATA SET MANAGEMENT with the TRIM bit
1385       set. In the SCSI command set (SBC-3) it is either the  UNMAP  or  WRITE
1386       SAME  command.  Note  there is no TRIM command however the term is fre‐
1387       quently used in the technical press.
1388
1389       Trim is a way of telling a storage device that  blocks  are  no  longer
1390       needed.   Keeping  the  pool of unwritten blocks large is important for
1391       the write performance of SSDs and the thrifty use of  real  storage  in
1392       thin  provisioned arrays. Currently file systems in recent OSes may is‐
1393       sue trims associated with file deletes. The trim option in ddpt may  be
1394       useful  when  a  partition or a whole SSD is to be "deleted". Note that
1395       ddpt is  bypassing  file  systems  in  that  it  only  offers  trim  on
1396       pass-through (pt) devices.
1397
1398       This  utility  issues  SCSI  commands to pt devices and for "trim" cur‐
1399       rently issues a SCSI WRITE SAME(16) command with the UNMAP bit set.  If
1400       the  pt  device  is  a SSD with a ATA interface then recent versions of
1401       Linux will translate the SCSI WRITE SAME to the ATA DATA SET MANAGEMENT
1402       command  with the TRIM bit set. The maximum size of each "trim" command
1403       sent is the size of the copy buffer (i.e. IBS * BPT  bytes).  And  that
1404       maximum can be reduced with the OBPC argument of the "bpt=" option.
1405
1406       The  trim  can  be  used various ways. One way is a copy where the copy
1407       buffer (or some part of it) is checked for zeros  as  is  done  by  the
1408       sparse oflag. When a zero segment is found, a trim "command" is sent to
1409       the OFILE. For example:
1410
1411          ddpt if=dsk.img bs=512 of=/dev/sdc oflag=pt,trim
1412
1413       The copy buffer is 64 KiB (since BPT  and  OBPC  default  to  128  when
1414       "bs=512")  and  it  is checked for all zeros. If it is all zeros then a
1415       trim command is sent to the corresponding location of /dev/sdc which is
1416       accessed via the pt interface. If it is not all zeros then a SCSI WRITE
1417       command is sent. Another way is to trim all or part of a disk. To  trim
1418       a whole disk (i.e. deleting all its data):
1419
1420          ddpt if=/dev/zero bs=512 of=/dev/sdc oflag=pt,trim
1421
1422       A  third  way  is to "self-trim" which is to only trim those parts of a
1423       disk that contain segments full of zeros:
1424
1425          ddpt    if=/dev/sdc    skip=0x2300     bs=512     iflag=pt,self,trim
1426       count=0x1234f0
1427
1428       The  "self"  oflag automatically sets up the output side of the copy to
1429       send trim commands  (if  required)  back  the  the  same  device  (i.e.
1430       /dev/sdc).  If this example was self-trimming a partition then the par‐
1431       tition would start at LBA 0x2300 and be 0x1234f0 blocks long.
1432
1433       Some random product examples: the Intel X25-M G2 SSDs  have  trim  with
1434       recent  firmware  and  they  do deterministic read zero after trim. The
1435       Seagate Pulsar SSD has an ATA interface which supports the  determinis‐
1436       tic  reads  of zero after the DATA SET MANAGEMENT command with the TRIM
1437       option.
1438

NVME SUPPORT

1440       The following information is Linux specific at this time. NVMe  devices
1441       in  Linux  have names like /dev/nvme0, /dev/nvme0n1 and /dev/nvme0n1p3.
1442       The first device name is a character device and some  "Admin"  commands
1443       can  be  sent to it (e.g. Identify) but no media access commands (which
1444       the NVMe specification calls the "NVM" Command set). The  number  given
1445       is  a  controller  identifier. Storage in NVMe is associated with name‐
1446       spaces which are numbered within a  controller,  starting  at  1  (e.g.
1447       /dev/nvme0n1  is  controller  0,  namespace  1). These device nodes are
1448       block devices and can be given as IFILE and/or OFILE. The third type of
1449       NVMe device node selects a partition (within a namespace, within a con‐
1450       troller). Partition numbers also start with 1.
1451
1452       By default ddpt will treat the second and third form  (of  NVMe  device
1453       nodes) as standard Linux block devices. So ddpt will act in the same as
1454       the dd utility would. In a similar fashion to accessing SCSI block  de‐
1455       vices  (e.g.  /dev/sdc3) get access NVMe block devices the "pt" flag is
1456       required, either with iflag=FLAGS and/or oflag=FLAGS. There is  a  SCSI
1457       to  NVMe Translation Layer (SNTL) in the sg3_utils library which under‐
1458       pins this utility.
1459

DD DIFFERENCES

1461       dd defaults "if=" and "of=" to stdin and stdout respectively. This fol‐
1462       lows  Unix filter conventions. However since dd and ddpt are often used
1463       to  read  binary  data  for   timing   purposes,   having   to   supply
1464       "of=/dev/null"  can  be  easily forgotten. Without it dd will typically
1465       spew binary data on the console. So ddpt has changed its defaults:  the
1466       "if=IFILE"  is  now  mandatory for direct copies and to read from stdin
1467       "if=-" can be used; "of=OFILE" remains optional but its default changes
1468       to "/dev/null" (or "NUL" in Windows). To send output to stdout ddpt ac‐
1469       cepts "of=-".
1470
1471       dd truncates OFILE unless "conv=notrunc" is given. When  dd  truncates,
1472       it truncates to zero length unless SEEK is greater than zero. ddpt does
1473       not truncate OFILE by default. If OFILE exists it will be  overwritten.
1474       The  overwrite  starts  at  block zero unless SEEK or "oflag=append" is
1475       given. If OFILE is a regular file then "oflag=trunc" (or  "conv=trunc")
1476       will truncate OFILE prior to the copy.
1477
1478       Numeric  arguments  to  ddpt can be given in hexadecimal, either with a
1479       leading "0x" or "0X" or with a  trailing  "h".  Note  that  dd  accepts
1480       "0x123"  but interprets it as "0 * 123" (i.e. zero). ddpt will also in‐
1481       terpret "x" as  multiplies  unless  the  left  operand  is  zero  (e.g.
1482       "0x123").   So   both  dd  and  ddpt  will  interpret  "skip=2x123"  as
1483       "skip=246".
1484
1485       Terabyte size disks make it impractical to copy all  the  data  into  a
1486       single buffer of 512 bytes length before writing it out. Therefore both
1487       dd and ddpt read a relatively small amount of  data  into  a  copy  (or
1488       transfer)  buffer  then write it out to the destination, repeating this
1489       process until the COUNT is exhausted.
1490
1491       A major difference in ddpt is the addition of BPT (Blocks Per Transfer)
1492       to control the size of the copy buffer. With dd, IBS is the size of the
1493       copy buffer and the unit of SKIP and COUNT. With ddpt, IBS * BPT is the
1494       size of the copy buffer and IBS is the unit of SKIP and COUNT. This al‐
1495       lows ddpt to have its IBS set to the logical block size of IFILE  with‐
1496       out  unduly  restricting  the  size of the copy buffer. And setting IBS
1497       (and OBS for OFILE) accurately is required when the pass-through inter‐
1498       face  is  used  since with the SCSI READ and WRITE commands the logical
1499       block size is implicit.
1500
1501       The way dd handles its copy buffer (outlined in  SUSv4  description  of
1502       dd)  is  relatively  complex, especially when IBS and OBS are different
1503       sizes. The restriction that ddpt places on IBS and OBS ( i.e. (((IBS  *
1504       BPT)  %  OBS) == 0) ) means that a single copy buffer can be used since
1505       its size is a multiple of both IBS and OBS. Being able to precisely de‐
1506       fine  the  copy buffer size in ddpt makes sparse writing, write sparing
1507       and trim operations simpler to define and the user to control.
1508
1509       ddpt does not support dd's "cbs=" option (conversion  block  size).  If
1510       the "cbs=" option is given to ddpt then it is ignored.
1511
1512       ddpt  adds  two  types  of  disk to disk, offloaded copies: XCOPY(LID1)
1513       first introduced in SPC-2 (standardized in 2001), and ODX  which  is  a
1514       subset  of  XCOPY(LID4)  first  introduced in SPC-4 draft (revision 34,
1515       2012).
1516

PROTECTION INFORMATION

1518       This section is about protection information which is typically an  ex‐
1519       tra  8  bytes  associated with each logical block. Those 8 byte are di‐
1520       vided into 3 fields: logical block guard (16 bit (2 byte) CRC), logical
1521       block  application tag (2 bytes) and the logical block reference tag (4
1522       bytes). The acronym DIF is sometimes used for protection information.
1523
1524       The feature to read and/or write protection information  by  using  the
1525       protect=RDP[,WRP]  option  is currently experimental. It should be used
1526       with care and may not "play well" with  some  other  features  such  as
1527       write  sparing  and sparse writing. It should be used to copy user data
1528       plus the associated protection information to or from a  regular  file.
1529       It could also be used for a device to device copy assuming the "pt" in‐
1530       terface is used for both. Also only modern SCSI disks  support  protec‐
1531       tion information.
1532
1533       When  RDP  or WRP is greater than 0 then a copy with associated protec‐
1534       tion information is active. In this state IBS and OBS must be the  same
1535       and  equal  to  the  logical block size of the device(s) formatted with
1536       protection information. If a SCSI disk with 512 byte logical block size
1537       has  protection information then the actual number of bytes transferred
1538       for each logical block is typically 520 bytes. For such a  disk  BS=512
1539       is required even when additional protection information is being trans‐
1540       ferred.
1541
1542       When protection type 2 is used, the "normal"  READ,  WRITE  and  VERIFY
1543       SCSI commands are disallowed. In this context "normal" means the 6, 10,
1544       12, and 16 byte variants. Only READ(32) and WRITE(32) can be used.  The
1545       32  byte  variants can be selected in this utility by using the operand
1546       'cdbsz=32'.
1547

MULTIPLIERS

1549       By default numeric arguments to options are assumed to be decimal.  Al‐
1550       most  all  numeric  arguments to options (e.g. COUNT in the count=COUNT
1551       option) may include one of these multiplicative suffixes: c C *1;  w  W
1552       *2;  b  B  *512;  k  K  KiB  *1,024;  KB *1,000; m M MiB *1,048,576; MB
1553       *1,000,000 . This pattern continues for "G", "T" and  "P".  The  latter
1554       two suffixes can only be used for 64 bit values. Some numeric arguments
1555       are limited to 32 bit values (e.g. BSin the bs=BS option).  Also a suf‐
1556       fix  of  the  form "x<n>" multiplies the leading number by <n>; however
1557       the combinations "0x" and "0X" are treated differently,  see  the  next
1558       paragraph.  These  multiplicative suffixes are compatible with GNU's dd
1559       command (since 2002) which claims compliance with the SI and  with  IEC
1560       60027-2 standards.
1561
1562       Alternatively numerical values can be given in hexadecimal indicated by
1563       either a leading "0x" or "0X", or by a trailing "h" or  "H".  When  hex
1564       numbers are given, suffix multipliers cannot be used.
1565
1566       If  a  numeric  argument is required to fit in 32 bits and is too large
1567       then an error is reported. Usually negative numbers are  not  permitted
1568       but  "count=-1"  is  a  special  case  and means "all available"; "ver‐
1569       bose=-1" is another special case.
1570

NOTES

1572       Copying data behind an Operating System's back can cause  problems.  In
1573       the    case    of    Linux,   users   should   look   at   this   link:
1574       https://linux-mm.org/Drop_Caches
1575       This command sequence may be useful:
1576         sync; echo 3 > /proc/sys/vm/drop_caches
1577
1578       A partial write is a write to the OFILE of less than  OBS  bytes.  This
1579       typically  occurs  at the end of a copy. dd can do partial writes. ddpt
1580       does partial writes to regular files and fifos (including stdout). How‐
1581       ever  ddpt  ignores partial writes when OFILE is a block device or a pt
1582       device. When ddpt ignores a partial write, it sends a  warning  to  the
1583       console (stderr).
1584
1585       At the end of the copy two lines are reported to the console:
1586          <in_full>+<in_partial> records in
1587          <out_full>+<out_partial> records out
1588
1589       The  "records  in" line is the number of full input blocks (each of IBS
1590       bytes) that have been read plus the number of partial blocks (  usually
1591       less than IBS bytes) that have been read. Following the lead of dd when
1592       'iflag=coe' is active a block that cannot be read (and has  zeros  sub‐
1593       stituted  for  its  output) is regarded as a partial read. The "records
1594       out" line is the number of full output blocks (each of OBS bytes)  that
1595       have  been written plus the number of partial blocks (usually less than
1596       OBS bytes) that have been written.
1597
1598       Block devices (e.g. /dev/sda and /dev/hda) can be given for IFILE.   If
1599       neither 'iflag=direct' nor 'iflag=pt' is given then normal block IO in‐
1600       volving buffering and caching is performed. If 'iflag=direct' is  given
1601       then  the buffering and caching is bypassed (this is applicable to both
1602       SCSI devices and ATA disks). When 'iflag=pt' is given SCSI commands are
1603       sent  to the device which bypasses most of the actions performed by the
1604       block layer.  The same applies for block devices given for OFILE.
1605
1606       All informative, warning and error reports are sent to stderr  so  that
1607       dd's output file can be stdout and remain unpolluted. If no options are
1608       given, then no copying (nor reading) takes place and a brief message is
1609       sent to stderr inviting the user to invoke ddpt again but with '--help'
1610       option to get the usage message.
1611
1612       Disk partition information can often be found with fdisk(8) [the  "-ul"
1613       argument  is  useful  in this respect]. Also parted(8) can be used like
1614       this: 'parted /dev/sda unit s print' .
1615
1616       For pt devices this utility issues SCSI READ and WRITE  (SBC)  commands
1617       which  are  appropriate  for  disks  and reading from CD/DVD/BD drives.
1618       Those commands are not formatted correctly for tape drives so ddpt can‐
1619       not  be  used  on tape drives via a pt device. If the largest block ad‐
1620       dress of the requested transfer exceeds a  32  bit  block  number  (i.e
1621       0xffffffff)  then a warning is issued and the pt device is accessed via
1622       SCSI READ(16) and WRITE(16) commands.
1623
1624       The attributes of a block device (e.g. partitions) are ignored when the
1625       pt  flag is used.  Hence the whole device is read (rather than just the
1626       second partition) by this invocation:
1627
1628          ddpt if=/dev/sdb2 iflag=pt of=t bs=512
1629
1630       Assuming /dev/sdb and /dev/sg2 refer to the same device, then after the
1631       following  two  invocations,  the  contents  of the files "t", "tt" and
1632       "ttt" should be same:
1633
1634          ddpt if=/dev/sdb of=tt bs=512
1635
1636          ddpt if=/dev/sg2 of=ttt bs=512
1637
1638       The SCSI READ(32) and WRITE(32) commands are restricted to  media  that
1639       is formatted with protection type 2. This is a T10 restriction.
1640

SIGNALS

1642       The  signal  handling  has been borrowed from GNU's dd: SIGINT, SIGQUIT
1643       and SIGPIPE report the number of remaining blocks to be transferred and
1644       the  records  in + out counts; then they have their default action. SI‐
1645       GUSR1 (or SIGINFO) causes the same information to  be  output  and  the
1646       copy continues.  All output caused by signals is sent to stderr.
1647
1648       Like  GNU's  dd,  ddpt  respects  the  signal  disposition of "ignored"
1649       (SIG_IGN) set by the shell, script or other program that invokes  ddpt.
1650       So in that case it will ignore such signals. Further dd ignores SIGUSR1
1651       if the environment variable POSIXLY_CORRECT is set  because  POSIX  de‐
1652       fines  dd will only act on SIGINFO (and Linux has no such signal); ddpt
1653       ignores the POSIXLY_CORRECT environment  variable.  As  recommended  by
1654       Susv3,  ddpt  does not expect the signal (blocking) mask to be blocking
1655       SIGUSR1 (SIGINFO), SIGINT or SIGPIPE on entry.
1656
1657       Unix system calls that do IO can be interrupted by  signal  processing,
1658       typically  returning  an  EINTR  error number. The dd utility (and many
1659       other Unix utilities) restart the IO operation  that  was  interrupted.
1660       While this will work most of the time for disk IO it is problematic for
1661       tape drives because the implicit position pointer on the tape may  have
1662       moved.   So  the  default  (i.e.  "intio=0") in this utility is to mask
1663       those signals during IO operations and only check them prior to  start‐
1664       ing  an  IO  operation.   Most low level IO (e.g. using SCSI command to
1665       write to a disk) will timeout if there is a low  level  error.  However
1666       NFS  (the  Network  File  System) will potentially wait for a long time
1667       (e.g. expecting a network problem will soon be fixed) and in this  case
1668       using "intio=1" may be best.
1669

VERIFY

1671       The  usual  way  to  check the two disks (or part of the disks) are the
1672       same is to move through the segments to be compared, reading from  both
1673       and  comparing  the returned buffers, stopping if there is an in equal‐
1674       ity.
1675
1676       This utility takes a different approach that relies on the OFILE  being
1677       a  pass  through  device. That pass-through device needs to support the
1678       SCSI VERIFY command with the BYTCHK field set to 1. Optionally, for the
1679       --prefetch option to improve performance that pass-through device needs
1680       to support the SCSI PRE-FETCH command with its IMMED bit set.
1681
1682       When the --verify option is given, instead of reading  both  IFILE  and
1683       OFILE,  only the IFILE is read. Then the result of that read is sent to
1684       the OFILE device as the data-out buffer of a VERIFY(BYTCHK=1)  command.
1685       So  the comparison is actually done on the OFILE device rather than the
1686       host computer's main memory.
1687
1688       If the --prefetch option is also given, then before the IFILE  read,  a
1689       PRE-FETCH(OFILE, IMMED) is sent. The IMMED bit will make it return more
1690       or less immediately. The effect of the PRE-FETCH should be to bring the
1691       contents  of  the data to be used for the OFILE side of the comparison,
1692       into the OFILE device's cache. And that  should  make  the  later  VER‐
1693       IFY(BYTCHK=1) command faster.
1694

TAPE

1696       There  is support for copies to and from tape drives in Linux. Only the
1697       st driver device names can be used (e.g. /dev/st0 and /dev/nst2). Hence
1698       use  of Linux pass-through device names (e.g. /dev/sg2) for tape drives
1699       is not supported. On Debian-based distributions, it is  suggested  that
1700       the  mt-st  package  is  installed as it provides a more fully-featured
1701       version of the "mt" tape control program.
1702
1703       Tape drives can operate in fixed- or variable-length  block  modes.  In
1704       variable-block  mode,  each  write to the tape writes a single block of
1705       that size. In fixed-block mode, each write to the tape must be a multi‐
1706       ple of the previously-selected block size.
1707
1708       The  block  size/mode  can be set with the mt command prior to invoking
1709       ddpt.  For example:
1710         # mt -f /dev/nst0 setblk 0
1711       sets variable-block mode, and
1712         # mt -f /dev/nst0 setblk 32768
1713       sets fixed-block mode with block size 32768 bytes.
1714
1715       Note that some tape drives support only fixed-block mode, and  possibly
1716       even only one block size. (For example, QIC-150 tapes use a fixed block
1717       size of 512 bytes.) There may also be restrictions on the  block  size,
1718       e.g.  it may have to be even.
1719
1720       When  using  ddpt to write to tape, if the final read from the input is
1721       less than OBS, it is padded to OBS bytes before writing to tape to  en‐
1722       sure  that  all  blocks  of the tape file are the same length. Having a
1723       shorter final block would fail if the drive is in fixed-block mode, and
1724       could create interchange problems. It is common to expect all blocks in
1725       a file on tape to be the same length. However, to tell ddpt to not  pad
1726       the final block, use 'oflag=nopad'.
1727
1728       The  st  tape  driver  normally  writes  a filemark when the file (e.g.
1729       /dev/nst0)  is  closed.  To  not  have  the   filemark   written,   use
1730       'oflag=nofm'.  One  use  case  for  that might be if using ddpt several
1731       times in succession to append more data to the same file  on  tape.  In
1732       that  case it is probably desirable to write the filemark at the end of
1733       the sequence. So either omit 'oflag=nofm' on the last ddpt  invocation,
1734       or manually write a filemark using mt after ddpt exits:
1735         # mt -f /dev/nst0 weof 1
1736
1737       For  reading from an unknown tape where the block size(s) is not known,
1738       read in variable-block mode specifying a large IBS. The st  driver  re‐
1739       turns  a  smaller  amount  of  data  if  the  size of the block read is
1740       smaller. Thus a command like:
1741         # ddpt if=/dev/nst0 of=output.bin bs=262144
1742       should read the file from tape regardless of the block size  used  (as‐
1743       suming  no  blocks  are  larger than 256KB). ddpt's verbose option will
1744       display what the actual block size(s) is.
1745

ENVIRONMENT VARIABLES

1747       If the command line invocation of an xcopy does not explicitly (and un‐
1748       ambiguously)  indicate whether the XCOPY SCSI command should be sent to
1749       IFILE (i.e.  the source) or OFILE (i.e. the destination) then  a  check
1750       is  made for the presence of the XCOPY_TO_SRC and XCOPY_TO_DST environ‐
1751       ment variables. If either one exists (but not both) then  it  indicates
1752       where the SCSI XCOPY command will be sent. By default the XCOPY command
1753       is sent to OFILE.
1754
1755       The ODX write from tokens variant is very complex to implement  if  the
1756       amount of data held in each ROD is not known. The value should be found
1757       in the "number of bytes represented" field in the ROD Token but that is
1758       not  well  supported yet by vendors. So for such cases, that number can
1759       be appended as a big endian 8 byte integer following each ROD Token  in
1760       the  RTF  file. The conv=rtf_len will cause that length to be appended.
1761       Specifying that option on each read to tokens and write from tokens in‐
1762       vocation   can   be   a  nuisance.  Setting  the  environment  variable
1763       ODX_RTF_LEN will cause this utility to act as if the  conv=rtf_len  op‐
1764       tion has been given.
1765
1766       Sometimes  the default block size of 512 can be a nuisance. This can be
1767       overridden by the value associated  with  the  DDPT_DEF_BS  environment
1768       variable. If the environment variable is not found, the value cannot be
1769       decoded or is zero or less, then the default block size remains at  512
1770       bytes.
1771

EXIT STATUS

1773       To  aid scripts that call ddpt, the exit status is set to indicate suc‐
1774       cess (0) or failure (1 or more). Note that some  of  the  lower  values
1775       correspond to the SCSI sense key values. The exit status values are:
1776
1777       0      success.  Also  conveys  boolean true for actions that result in
1778              true or false (e.g. sgl equality tests)
1779
1780       1      syntax error. Either illegal command line options, options  with
1781              bad arguments or a combination of options that is not permitted.
1782
1783       2      the  device  reports  that it is not ready for the operation re‐
1784              quested.  The device may be in the  process  of  becoming  ready
1785              (e.g.  spinning up but not at speed) so the utility may work af‐
1786              ter a wait.
1787
1788       3      the device reports a  medium  or  hardware  error  (or  a  blank
1789              check).  For  example  an attempt to read a corrupted block on a
1790              disk will yield this value.
1791
1792       5      the device reports an "illegal request" with an additional sense
1793              code  other  than "invalid operation code". This is often a sup‐
1794              ported command with a field set requesting an unsupported  capa‐
1795              bility.
1796
1797       6      the  device  reports  a "unit attention" condition. This usually
1798              indicates that something unrelated to the requested command  has
1799              occurred  (e.g.  a  device reset) potentially before the current
1800              SCSI command was sent. The requested command has not  been  exe‐
1801              cuted  by  the  device.  Note that unit attention conditions are
1802              usually only reported once by a device.
1803
1804       7      the device reports a "data protect" sense key. This implies some
1805              mechanism  has blocked writes (or possibly all access to the me‐
1806              dia).
1807
1808       9      the device reports an illegal request with an  additional  sense
1809              code  of  "invalid  operation  code" which means that it doesn't
1810              support the requested command.
1811
1812       10     the device reports a "copy aborted". This implies  another  com‐
1813              mand  or  device problem has stopped and copy operation. The EX‐
1814              TENDED COPY family of commands (including WRITE USING TOKEN) may
1815              return this sense key.
1816
1817       11     the  device  reports  an  aborted command. In some cases aborted
1818              commands can be  retried  immediately  (e.g.  if  the  transport
1819              aborted the command due to congestion).
1820
1821       14     the  DEVICE  reports  a miscompare sense key. VERIFY and COMPARE
1822              AND WRITE commands may report this.
1823
1824       15     the utility is unable to open, close or  use  the  given  IFILE,
1825              OFILE  or  other file. The given file name could be incorrect or
1826              there may be permission problems. Adding the -v option may  give
1827              more information.
1828
1829       20     the  device reports it has a check condition but "no sense".  It
1830              is unlikely that this value will occur as an exit status.
1831
1832       21     the device reports a "recovered error".  The  requested  command
1833              was  successful.   Most likely a utility will report a recovered
1834              error to stderr and continue, probably leaving the utility  with
1835              an exit status of 0 .
1836
1837       24     the device reports a SCSI status of "reservation conflict". This
1838              means access to the device with the  current  command  has  been
1839              blocked  because another machine (HBA or SCSI "initiator") holds
1840              a reservation on this device. On modern SCSI systems this is re‐
1841              lated  to  the  use of the PERSISTENT RESERVATION family of com‐
1842              mands.
1843
1844       25     the DEVICE reports a SCSI status of "condition  met".  Currently
1845              only the PRE-FETCH command (see SBC-4) yields this status.
1846
1847       26     the  DEVICE  reports a SCSI status of "busy". SAM-5 defines this
1848              status as the logical unit is temporarily unable  to  process  a
1849              command.  It is recommended to re-issue the command.
1850
1851       27     the DEVICE reports a SCSI status of "task set full".
1852
1853       28     the  DEVICE  reports a SCSI status of "ACA active". ACA is "auto
1854              contingent allegiance" and is seldom used.
1855
1856       29     the DEVICE reports a SCSI status of "task aborted". SAM-5  says:
1857              "This status shall be returned if a command is aborted by a com‐
1858              mand or task management function on another I_T  nexus  and  the
1859              Control mode page TAS bit is set to one".
1860
1861       31     error  involving  two  or more command line options. Either they
1862              contradict or select an unsupported mode.
1863
1864       32     the is a logic error in the utility. It corresponds to code com‐
1865              ments like "shouldn't/can't get here". Perhaps the author should
1866              be contacted.
1867
1868       33     the command sent to device has timed out. This occurs  in  Linux
1869              only;  in  other ports a command timeout will appear as a trans‐
1870              port (or OS) error.
1871
1872       36     no error has occurred. For actions that  result  in  a  boolean,
1873              this exit status indicates false.
1874
1875       40     the  command  sent to a device has received an "aborted command"
1876              sense key with an additional sense code of 0x10. This  value  is
1877              related to problems with protection information (PI or DIF). For
1878              example this error may occur when reading a  block  on  a  drive
1879              that  has  never been written (or is unmapped) if that drive was
1880              formatted with type 1, 2 or 3 protection.
1881
1882       48     this is an internal message indicating a NVMe status field  (SF)
1883              is other than zero after a command has been executed (i.e. some‐
1884              thing went wrong).  Work in this area is currently experimental.
1885
1886       49     low level driver reports a response's residual count (i.e.  num‐
1887              ber  of  bytes  actually  received  by HBA is 'requested_bytes -
1888              residual_count') that is too high. So no useful  processing  can
1889              be done with that response.
1890
1891       50 + <os_error_number>
1892              OS system calls that fail often return a small integer number to
1893              help indicate what the error is. For example in Unix the inabil‐
1894              ity  of  a  system  call to allocate memory returns (in 'errno')
1895              ENOMEM which often is associated with  the  integer  12.  So  62
1896              (i.e. '50 + 12') may be returned by a utility in this case.
1897
1898       90     the flock flag has been given on a device and some other process
1899              holds the advisory exclusive lock.
1900
1901       97     the response to a SCSI command failed sanity checks.
1902
1903       98     the device reports it  has  a  check  condition  but  the  error
1904              doesn't fit into any of the above categories.
1905
1906       99     any  errors  that  can't  be categorized into values 1 to 98 may
1907              yield this value. This includes transport and  operating  system
1908              errors after the command has been sent to the device.
1909
1910       100    a command received a 'parameter list length error'.
1911
1912       101    a  command  received 'illegal field in parameter list'. This may
1913              occur with an odx copy if some combination of parameters is  il‐
1914              legal or not supported (e.g.  iflag=immed).
1915
1916       105    a  command received 'operation in progress'. This may occur with
1917              an odx copy when the given LID is already being used by  another
1918              process  (e.g.  also  using odx) on the same machine. Choose an‐
1919              other LID.
1920
1921       110    a command received  'invalid  token  operation,  cause  not  re‐
1922              portable'.  This  may occur with an odx operation when the given
1923              ROD Token is invalid. One reason for that may be the  inactivity
1924              timeout  has been reached and the copy manager has cancelled the
1925              ROD Token.
1926
1927       110 + <asc_23h_ascq_code>
1928              these status values provide more information  than  exit  status
1929              110.  See  SPC-5  ASC  and  ASCQ assignments (currently in Annex
1930              F.2), specifically the entries for asc=23h .  For  example  exit
1931              status 112 corresponds to asc=23h, ascq=2h which implies the odx
1932              copy manager does not support copies between  LUs  in  different
1933              targets.  That  is  optional; an odx copy manager is required to
1934              support copies between LUs (that are block devices) in the  same
1935              target.
1936
1937       126    the  utility was found but could not be executed. That might oc‐
1938              cur if the executable does not have execute permissions.
1939
1940       127    This is the exit status for utility not found. That might  occur
1941              when a script calls a utility in this package but the PATH envi‐
1942              ronment variable has not been properly set  up,  so  the  script
1943              cannot find the executable.
1944
1945       128 + <signum>
1946              If a signal kills a utility then the exit status is 128 plus the
1947              signal number. For example if a segmentation fault occurs then a
1948              utility is typically killed by SIGSEGV which according to 'man 7
1949              signal' has an associated signal number of 11; so the exit  sta‐
1950              tus will be 139 .
1951
1952       255    the utility tried to yield an exit status of 255 or larger. That
1953              should not happen; given here for completeness.
1954

EXAMPLES

1956       The examples in this page use Linux device names. For  suitable  device
1957       names   in  other  supported  Operating  Systems  see  this  web  page:
1958       https://sg.danny.cz/sg/device_name.html . The sg3_utils(8) man page  in
1959       the sg3_utils package also covers device naming.
1960
1961       ddpt usage looks quite similar to dd:
1962
1963          ddpt if=/dev/sg0 of=t bs=512 count=1MB
1964
1965       This  will  copy  1  million 512 byte blocks from the device associated
1966       with /dev/sg0 (which should have 512 byte blocks) to a file  called  t.
1967       Assuming  /dev/sda  and  /dev/sg0 are the same device then the above is
1968       equivalent to:
1969
1970          dd if=/dev/sda iflag=direct of=t bs=512 count=1000000
1971
1972       although dd's speed may improve if bs was larger and count was suitably
1973       reduced.  The  use  of the 'iflag=direct' option bypasses the buffering
1974       and caching that is usually done on a block device.
1975
1976       The dd command's bs argument can be thought of as roughly equivalent to
1977       ddpt's  bs*bpt . dd almost assumes buffering on a block device and will
1978       work as long as bs is a multiple of  the  actual  logical  block  size.
1979       Since ddpt can work at a lower level in some cases the bs argument must
1980       be a disk's actual logical block size. Thus the bpt argument was intro‐
1981       duced  to  make  the  copy more efficient. So these two invocations are
1982       roughly equivalent:
1983
1984          dd if=/dev/sda of=t bs=8k count=64
1985          ddpt if=/dev/sda of=t bs=512 bpt=16 count=1k
1986
1987       In both cases the total number of bytes moved is bs*count  .  And  that
1988       will  be done by reading 8k (8192 bytes) into a buffer then writing out
1989       that buffer to the file t. The read write sequence continues until  the
1990       count is complete or an error occurs.
1991
1992       The  'of2=' option can save time when the input would otherwise need to
1993       be read twice. For example, to copy data and take a md5sum of it  with‐
1994       out needing to re-read the data:
1995
1996         mkfifo fif
1997         md5sum fif &
1998         ddpt if=/dev/sg3 iflag=coe of=sg3.img oflag=sparse of2=fif bs=512
1999
2000       This  will  image  /dev/sg3 (e.g. an unmounted disk) and place the con‐
2001       tents in the (sparse) file sg3.img . Without  re-reading  the  data  it
2002       will also perform a md5sum calculation on the image.
2003
2004       Now  we use sparse writing logic to get some idea of how many blocks on
2005       a disk are full of zeros. After a SCSI FORMAT UNIT command  or  an  ATA
2006       SECURITY ERASE command a disk may be all zeros.
2007
2008          ddpt if=/dev/sdc bs=512 oflag=sparse
2009
2010       Since  no "of=" option is given, output goes to /dev/null so nothing is
2011       actually written so the "records out" will be zero. However there  will
2012       be  a  count of "records in" and "bypassed records out". If /dev/sdc is
2013       full of zeros then "records in" and "bypassed records out" will be  the
2014       same. Since the "bpt=" option is not given it defaults to "bpt=128,128"
2015       so the copy buffer will be 64 KiB and the sparse check for  zeros  will
2016       be done with 64 KiB (128 block) granularity.
2017
2018       For examples of the trim and self,trim options see the section above on
2019       TRIM, UNMAP AND WRITE SAME.
2020
2021       Following is an example run on a Windows OS using the '--wscan'  option
2022       which  shows  the  available device names (e.g. PD1) and the associated
2023       volume name(s):
2024
2025          ddpt -w
2026       PD0     [C]     FUJITSU   MHY2160BH         0000
2027       PD1     [DF]    WD        2500BEV External  1.05  WD-WXE90
2028       CDROM0  [E]     MATSHITA DVD/CDRW UJDA775  CB03
2029
2030       So, for example, volumes D: and F: reside on PhysicalDisk1 (abbreviated
2031       to "PD1") which is manufactured by WD (Western Digital).
2032
2033       Further    examples    can    be    found    on    this    web    page:
2034       https://sg.danny.cz/sg/ddpt.html . There is a text file containing  ex‐
2035       amples  called  ddpt_examples.txt  in the "doc" directory of this pack‐
2036       age's distribution tarball. The ddpt_examples.txt  file  contains  some
2037       examples of using job files.
2038

AUTHORS

2040       Written by Doug Gilbert
2041

REPORTING BUGS

2043       Report bugs to <dgilbert at interlog dot com>.
2044
2046       Copyright © 2008-2021 Douglas Gilbert
2047       This  software is distributed under the GPL version 2. There is NO war‐
2048       ranty; not even for MERCHANTABILITY or FITNESS FOR  A  PARTICULAR  PUR‐
2049       POSE.
2050

SEE ALSO

2052       This utility has companion/helper utilities ddptctl(8), ddpt_sgl(8)
2053       There is a web page discussing ddpt at https://sg.danny.cz/sg/ddpt.html
2054
2055       The lmbench package contains lmdd which is also interesting. For moving
2056       data to and from tapes  see  dt  which  is  found  at  http://www.scsi
2057       faq.org/RMiller_Tools/index.html
2058
2059       To change mode parameters that effect a SCSI device's caching and error
2060       recovery see sdparm(sdparm)
2061
2062       To verify the data on the media is readable see: sg_verify(sg3_utils)
2063
2064       To scan and repair disk partitions see TestDisk (testdisk).
2065
2066       Additional references: dd(1), open(2),  flock(2),  sg_xcopy,sg_copy_re‐
2067       sults, sg_dd(sg3_utils)
2068
2069
2070
2071ddpt-0.97                         April 2021                           DDPT(8)
Impressum