coreutils: dd invocation
1
1 11.2 ‘dd’: Convert and copy a file
1 ==================================
1
1 ‘dd’ copies a file (from standard input to standard output, by default)
1 with a changeable I/O block size, while optionally performing
1 conversions on it. Synopses:
1
1 dd [OPERAND]...
1 dd OPTION
1
11 The only options are ‘--help’ and ‘--version’. ⇒Common
options. ‘dd’ accepts the following operands, whose syntax was
1 inspired by the DD (data definition) statement of OS/360 JCL.
1
1 ‘if=FILE’
1 Read from FILE instead of standard input.
1
1 ‘of=FILE’
1 Write to FILE instead of standard output. Unless ‘conv=notrunc’ is
1 given, ‘dd’ truncates FILE to zero bytes (or the size specified
1 with ‘seek=’).
1
1 ‘ibs=BYTES’
1 Set the input block size to BYTES. This makes ‘dd’ read BYTES per
1 block. The default is 512 bytes.
1
1 ‘obs=BYTES’
1 Set the output block size to BYTES. This makes ‘dd’ write BYTES
1 per block. The default is 512 bytes.
1
1 ‘bs=BYTES’
1 Set both input and output block sizes to BYTES. This makes ‘dd’
1 read and write BYTES per block, overriding any ‘ibs’ and ‘obs’
1 settings. In addition, if no data-transforming ‘conv’ option is
1 specified, input is copied to the output as soon as it’s read, even
1 if it is smaller than the block size.
1
1 ‘cbs=BYTES’
1 Set the conversion block size to BYTES. When converting
1 variable-length records to fixed-length ones (‘conv=block’) or the
1 reverse (‘conv=unblock’), use BYTES as the fixed record length.
1
1 ‘skip=N’
1 Skip N ‘ibs’-byte blocks in the input file before copying. If
1 ‘iflag=skip_bytes’ is specified, N is interpreted as a byte count
1 rather than a block count.
1
1 ‘seek=N’
1 Skip N ‘obs’-byte blocks in the output file before copying. if
1 ‘oflag=seek_bytes’ is specified, N is interpreted as a byte count
1 rather than a block count.
1
1 ‘count=N’
1 Copy N ‘ibs’-byte blocks from the input file, instead of everything
1 until the end of the file. if ‘iflag=count_bytes’ is specified, N
1 is interpreted as a byte count rather than a block count. Note if
1 the input may return short reads as could be the case when reading
1 from a pipe for example, ‘iflag=fullblock’ will ensure that
1 ‘count=’ corresponds to complete input blocks rather than the
1 traditional POSIX specified behavior of counting input read
1 operations.
1
1 ‘status=LEVEL’
1 Transfer information is normally output to stderr upon receipt of
1 the ‘INFO’ signal or when ‘dd’ exits. Specifying LEVEL will adjust
1 the amount of information printed, with the last LEVEL specified
1 taking precedence.
1
1 ‘none’
1 Do not print any informational or warning messages to stderr.
1 Error messages are output as normal.
1
1 ‘noxfer’
1 Do not print the final transfer rate and volume statistics
1 that normally make up the last status line.
1
1 ‘progress’
1 Print the transfer rate and volume statistics on stderr, when
1 processing each input block. Statistics are output on a
1 single line at most once every second, but updates can be
1 delayed when waiting on I/O.
1
1 ‘conv=CONVERSION[,CONVERSION]...’
1 Convert the file as specified by the CONVERSION argument(s). (No
1 spaces around any comma(s).)
1
1 Conversions:
1
1 ‘ascii’
1 Convert EBCDIC to ASCII, using the conversion table specified
1 by POSIX. This provides a 1:1 translation for all 256 bytes.
1 This option implies ‘conv=unblock’; input is converted to
1 ASCII before trailing spaces are deleted.
1
1 ‘ebcdic’
1 Convert ASCII to EBCDIC. This is the inverse of the ‘ascii’
1 conversion. This option implies ‘conv=block’; trailing spaces
1 are added before being converted to EBCDIC.
1
1 ‘ibm’
1 This acts like ‘conv=ebcdic’, except it uses the alternate
1 conversion table specified by POSIX. This is not a 1:1
1 translation, but reflects common historical practice for ‘~’,
1 ‘[’, and ‘]’.
1
1 The ‘ascii’, ‘ebcdic’, and ‘ibm’ conversions are mutually
1 exclusive. If you use any of these options, you should also
1 use the ‘cbs=’ option.
1
1 ‘block’
1 For each line in the input, output ‘cbs’ bytes, replacing the
1 input newline with a space and padding with spaces as
1 necessary.
1
1 ‘unblock’
1 Remove any trailing spaces in each ‘cbs’-sized input block,
1 and append a newline.
1
1 The ‘block’ and ‘unblock’ conversions are mutually exclusive.
1
1 ‘lcase’
1 Change uppercase letters to lowercase.
1
1 ‘ucase’
1 Change lowercase letters to uppercase.
1
1 The ‘lcase’ and ‘ucase’ conversions are mutually exclusive.
1
1 ‘sparse’
1 Try to seek rather than write NUL output blocks. On a file
1 system that supports sparse files, this will create sparse
1 output when extending the output file. Be careful when using
1 this option in conjunction with ‘conv=notrunc’ or
1 ‘oflag=append’. With ‘conv=notrunc’, existing data in the
1 output file corresponding to NUL blocks from the input, will
1 be untouched. With ‘oflag=append’ the seeks performed will be
1 ineffective. Similarly, when the output is a device rather
1 than a file, NUL input blocks are not copied, and therefore
1 this option is most useful with virtual or pre zeroed devices.
1
1 ‘swab’
1 Swap every pair of input bytes. GNU ‘dd’, unlike others,
1 works when an odd number of bytes are read—the last byte is
1 simply copied (since there is nothing to swap it with).
1
1 ‘sync’
1 Pad every input block to size of ‘ibs’ with trailing zero
1 bytes. When used with ‘block’ or ‘unblock’, pad with spaces
1 instead of zero bytes.
1
1 The following “conversions” are really file flags and don’t affect
1 internal processing:
1
1 ‘excl’
1 Fail if the output file already exists; ‘dd’ must create the
1 output file itself.
1
1 ‘nocreat’
1 Do not create the output file; the output file must already
1 exist.
1
1 The ‘excl’ and ‘nocreat’ conversions are mutually exclusive.
1
1 ‘notrunc’
1 Do not truncate the output file.
1
1 ‘noerror’
1 Continue after read errors.
1
1 ‘fdatasync’
1 Synchronize output data just before finishing. This forces a
1 physical write of output data.
1
1 ‘fsync’
1 Synchronize output data and metadata just before finishing.
1 This forces a physical write of output data and metadata.
1
1 ‘iflag=FLAG[,FLAG]...’
1 Access the input file using the flags specified by the FLAG
1 argument(s). (No spaces around any comma(s).)
1
1 ‘oflag=FLAG[,FLAG]...’
1 Access the output file using the flags specified by the FLAG
1 argument(s). (No spaces around any comma(s).)
1
1 Here are the flags. Not every flag is supported on every operating
1 system.
1
1 ‘append’
1 Write in append mode, so that even if some other process is
1 writing to this file, every ‘dd’ write will append to the
1 current contents of the file. This flag makes sense only for
1 output. If you combine this flag with the ‘of=FILE’ operand,
1 you should also specify ‘conv=notrunc’ unless you want the
1 output file to be truncated before being appended to.
1
1 ‘cio’
1 Use concurrent I/O mode for data. This mode performs direct
1 I/O and drops the POSIX requirement to serialize all I/O to
1 the same file. A file cannot be opened in CIO mode and with a
1 standard open at the same time.
1
1 ‘direct’
1 Use direct I/O for data, avoiding the buffer cache. Note that
1 the kernel may impose restrictions on read or write buffer
1 sizes. For example, with an ext4 destination file system and
1 a Linux-based kernel, using ‘oflag=direct’ will cause writes
1 to fail with ‘EINVAL’ if the output buffer size is not a
1 multiple of 512.
1
1 ‘directory’
1
1 Fail unless the file is a directory. Most operating systems
1 do not allow I/O to a directory, so this flag has limited
1 utility.
1
1 ‘dsync’
1 Use synchronized I/O for data. For the output file, this
1 forces a physical write of output data on each write. For the
1 input file, this flag can matter when reading from a remote
1 file that has been written to synchronously by some other
1 process. Metadata (e.g., last-access and last-modified time)
1 is not necessarily synchronized.
1
1 ‘sync’
1 Use synchronized I/O for both data and metadata.
1
1 ‘nocache’
1 Request to discard the system data cache for a file. When
1 count=0 all cached data for the file is specified, otherwise
1 the cache is dropped for the processed portion of the file.
1 Also when count=0, failure to discard the cache is diagnosed
1 and reflected in the exit status.
1
1 Note data that is not already persisted to storage will not be
1 discarded from cache, so note the use of the “sync” options in
1 the examples below, which are used to maximize the
1 effectiveness of the ‘nocache’ flag.
1
1 Here are some usage examples:
1
1 # Advise to drop cache for whole file
1 dd if=ifile iflag=nocache count=0
1
1 # Ensure drop cache for the whole file
1 dd of=ofile oflag=nocache conv=notrunc,fdatasync count=0
1
1 # Advise to drop cache for part of file
1 # Note the kernel will only consider complete and
1 # already persisted pages.
1 dd if=ifile iflag=nocache skip=10 count=10 of=/dev/null
1
1 # Stream data using just the read-ahead cache.
1 # See also the ‘direct’ flag.
1 dd if=ifile of=ofile iflag=nocache oflag=nocache,sync
1
1 ‘nonblock’
1 Use non-blocking I/O.
1
1 ‘noatime’
11 Do not update the file’s access timestamp. ⇒File
timestamps. Some older file systems silently ignore this
1 flag, so it is a good idea to test it on your files before
1 relying on it.
1
1 ‘noctty’
1 Do not assign the file to be a controlling terminal for ‘dd’.
1 This has no effect when the file is not a terminal. On many
1 hosts (e.g., GNU/Linux hosts), this option has no effect at
1 all.
1
1 ‘nofollow’
1 Do not follow symbolic links.
1
1 ‘nolinks’
1 Fail if the file has multiple hard links.
1
1 ‘binary’
1 Use binary I/O. This option has an effect only on nonstandard
1 platforms that distinguish binary from text I/O.
1
1 ‘text’
1 Use text I/O. Like ‘binary’, this option has no effect on
1 standard platforms.
1
1 ‘fullblock’
1 Accumulate full blocks from input. The ‘read’ system call may
1 return early if a full block is not available. When that
1 happens, continue calling ‘read’ to fill the remainder of the
1 block. This flag can be used only with ‘iflag’. This flag is
1 useful with pipes for example as they may return short reads.
1 In that case, this flag is needed to ensure that a ‘count=’
1 argument is interpreted as a block count rather than a count
1 of read operations.
1
1 ‘count_bytes’
1 Interpret the ‘count=’ operand as a byte count, rather than a
1 block count, which allows specifying a length that is not a
1 multiple of the I/O block size. This flag can be used only
1 with ‘iflag’.
1
1 ‘skip_bytes’
1 Interpret the ‘skip=’ operand as a byte count, rather than a
1 block count, which allows specifying an offset that is not a
1 multiple of the I/O block size. This flag can be used only
1 with ‘iflag’.
1
1 ‘seek_bytes’
1 Interpret the ‘seek=’ operand as a byte count, rather than a
1 block count, which allows specifying an offset that is not a
1 multiple of the I/O block size. This flag can be used only
1 with ‘oflag’.
1
1 These flags are not supported on all systems, and ‘dd’ rejects
1 attempts to use them when they are not supported. When reading
1 from standard input or writing to standard output, the ‘nofollow’
1 and ‘noctty’ flags should not be specified, and the other flags
1 (e.g., ‘nonblock’) can affect how other processes behave with the
1 affected file descriptors, even after ‘dd’ exits.
1
1 The numeric-valued strings above (N and BYTES) can be followed by a
1 multiplier: ‘b’=512, ‘c’=1, ‘w’=2, ‘xM’=M, or any of the standard block
1 size suffixes like ‘k’=1024 (⇒Block size).
1
1 Any block size you specify via ‘bs=’, ‘ibs=’, ‘obs=’, ‘cbs=’ should
1 not be too large—values larger than a few megabytes are generally
1 wasteful or (as in the gigabyte..exabyte case) downright
1 counterproductive or error-inducing.
1
1 To process data that is at an offset or size that is not a multiple
1 of the I/O block size, you can use the ‘skip_bytes’, ‘seek_bytes’ and
1 ‘count_bytes’ flags. Alternatively the traditional method of separate
1 ‘dd’ invocations can be used. For example, the following shell commands
1 copy data in 512 KiB blocks between a disk and a tape, but do not save
1 or restore a 4 KiB label at the start of the disk:
1
1 disk=/dev/rdsk/c0t1d0s2
1 tape=/dev/rmt/0
1
1 # Copy all but the label from disk to tape.
1 (dd bs=4k skip=1 count=0 && dd bs=512k) <$disk >$tape
1
1 # Copy from tape back to disk, but leave the disk label alone.
1 (dd bs=4k seek=1 count=0 && dd bs=512k) <$tape >$disk
1
1 For failing disks, other tools come with a great variety of extra
1 functionality to ease the saving of as much data as possible before the
1 disk finally dies, e.g. GNU ‘ddrescue’
1 (https://www.gnu.org/software/ddrescue/). However, in some cases such a
1 tool is not available or the administrator feels more comfortable with
1 the handling of ‘dd’. As a simple rescue method, call ‘dd’ as shown in
1 the following example: the options ‘conv=noerror,sync’ are used to
1 continue after read errors and to pad out bad reads with NULs, while
1 ‘iflag=fullblock’ caters for short reads (which traditionally never
1 occur on disk based devices):
1
1 # Rescue data from an (unmounted!) partition of a failing disk.
1 dd conv=noerror,sync iflag=fullblock </dev/sda1 > /mnt/rescue.img
1
1 Sending an ‘INFO’ signal (or ‘USR1’ signal where that is unavailable)
1 to a running ‘dd’ process makes it print I/O statistics to standard
1 error and then resume copying. In the example below, ‘dd’ is run in the
1 background to copy 5GB of data. The ‘kill’ command makes it output
1 intermediate I/O statistics, and when ‘dd’ completes normally or is
1 killed by the ‘SIGINT’ signal, it outputs the final statistics.
1
1 # Ignore the signal so we never inadvertently terminate the dd child.
1 # Note this is not needed when SIGINFO is available.
1 trap '' USR1
1
1 # Run dd with the fullblock iflag to avoid short reads
1 # which can be triggered by reception of signals.
1 dd iflag=fullblock if=/dev/zero of=/dev/null count=5000000 bs=1000 & pid=$!
1
1 # Output stats every second.
1 while kill -s USR1 $pid 2>/dev/null; do sleep 1; done
1
1 The above script will output in the following format:
1
1 3441325+0 records in
1 3441325+0 records out
1 3441325000 bytes (3.4 GB, 3.2 GiB) copied, 1.00036 s, 3.4 GB/s
1 5000000+0 records in
1 5000000+0 records out
1 5000000000 bytes (5.0 GB, 4.7 GiB) copied, 1.44433 s, 3.5 GB/s
1
1 The ‘status=progress’ option periodically updates the last line of
1 the transfer statistics above.
1
1 On systems lacking the ‘INFO’ signal ‘dd’ responds to the ‘USR1’
1 signal instead, unless the ‘POSIXLY_CORRECT’ environment variable is
1 set.
1
1 An exit status of zero indicates success, and a nonzero value
1 indicates failure.
1