coreutils: tee invocation
1
1 17.1 ‘tee’: Redirect output to multiple files or processes
1 ==========================================================
1
1 The ‘tee’ command copies standard input to standard output and also to
1 any files given as arguments. This is useful when you want not only to
1 send some data down a pipe, but also to save a copy. Synopsis:
1
1 tee [OPTION]... [FILE]...
1
1 If a file being written to does not already exist, it is created. If
1 a file being written to already exists, the data it previously contained
1 is overwritten unless the ‘-a’ option is used.
1
1 In previous versions of GNU coreutils (v5.3.0 - v8.23), a FILE of ‘-’
1 caused ‘tee’ to send another copy of input to standard output. However,
1 as the interleaved output was not very useful, ‘tee’ now conforms to
1 POSIX which explicitly mandates it to treat ‘-’ as a file with such
1 name.
1
11 The program accepts the following options. Also see ⇒Common
options.
1
1 ‘-a’
1 ‘--append’
1 Append standard input to the given files rather than overwriting
1 them.
1
1 ‘-i’
1 ‘--ignore-interrupts’
1 Ignore interrupt signals.
1
1 ‘-p’
1 ‘--output-error[=MODE]’
1 Adjust the behavior with errors on the outputs, with the long form
1 option supporting selection between the following MODEs:
1
1 ‘warn’
1 Warn on error opening or writing any output, including pipes.
1 Writing is continued to still open files/pipes. Exit status
1 indicates failure if any output has an error.
1
1 ‘warn-nopipe’
1 This is the default MODE when not specified, or when the short
1 form ‘-p’ is used. Warn on error opening or writing any
1 output, except pipes. Writing is continued to still open
1 files/pipes. Exit status indicates failure if any non pipe
1 output had an error.
1
1 ‘exit’
1 Exit on error opening or writing any output, including pipes.
1
1 ‘exit-nopipe’
1 Exit on error opening or writing any output, except pipes.
1
1 The ‘tee’ command is useful when you happen to be transferring a
1 large amount of data and also want to summarize that data without
1 reading it a second time. For example, when you are downloading a DVD
1 image, you often want to verify its signature or checksum right away.
1 The inefficient way to do it is simply:
1
1 wget https://example.com/some.iso && sha1sum some.iso
1
1 One problem with the above is that it makes you wait for the download
1 to complete before starting the time-consuming SHA1 computation.
1 Perhaps even more importantly, the above requires reading the DVD image
1 a second time (the first was from the network).
1
1 The efficient way to do it is to interleave the download and SHA1
1 computation. Then, you’ll get the checksum for free, because the entire
1 process parallelizes so well:
1
1 # slightly contrived, to demonstrate process substitution
1 wget -O - https://example.com/dvd.iso \
1 | tee >(sha1sum > dvd.sha1) > dvd.iso
1
1 That makes ‘tee’ write not just to the expected output file, but also
1 to a pipe running ‘sha1sum’ and saving the final checksum in a file
1 named ‘dvd.sha1’.
1
1 Note, however, that this example relies on a feature of modern shells
11 called “process substitution” (the ‘>(command)’ syntax, above; ⇒
Process Substitution (bash)Process Substitution.), so it works with
1 ‘zsh’, ‘bash’, and ‘ksh’, but not with ‘/bin/sh’. So if you write code
1 like this in a shell script, be sure to start the script with
1 ‘#!/bin/bash’.
1
1 Note also that if any of the process substitutions (or piped stdout)
1 might exit early without consuming all the data, the ‘-p’ option is
1 needed to allow ‘tee’ to continue to process the input to any remaining
1 outputs.
1
1 Since the above example writes to one file and one process, a more
1 conventional and portable use of ‘tee’ is even better:
1
1 wget -O - https://example.com/dvd.iso \
1 | tee dvd.iso | sha1sum > dvd.sha1
1
1 You can extend this example to make ‘tee’ write to two processes,
1 computing MD5 and SHA1 checksums in parallel. In this case, process
1 substitution is required:
1
1 wget -O - https://example.com/dvd.iso \
1 | tee >(sha1sum > dvd.sha1) \
1 >(md5sum > dvd.md5) \
1 > dvd.iso
1
1 This technique is also useful when you want to make a _compressed_
1 copy of the contents of a pipe. Consider a tool to graphically
1 summarize disk usage data from ‘du -ak’. For a large hierarchy, ‘du
1 -ak’ can run for a long time, and can easily produce terabytes of data,
1 so you won’t want to rerun the command unnecessarily. Nor will you want
1 to save the uncompressed output.
1
1 Doing it the inefficient way, you can’t even start the GUI until
1 after you’ve compressed all of the ‘du’ output:
1
1 du -ak | gzip -9 > /tmp/du.gz
1 gzip -d /tmp/du.gz | xdiskusage -a
1
1 With ‘tee’ and process substitution, you start the GUI right away and
1 eliminate the decompression completely:
1
1 du -ak | tee >(gzip -9 > /tmp/du.gz) | xdiskusage -a
1
1 Finally, if you regularly create more than one type of compressed
1 tarball at once, for example when ‘make dist’ creates both
1 ‘gzip’-compressed and ‘bzip2’-compressed tarballs, there may be a better
1 way. Typical ‘automake’-generated ‘Makefile’ rules create the two
1 compressed tar archives with commands in sequence, like this (slightly
1 simplified):
1
1 tardir=your-pkg-M.N
1 tar chof - "$tardir" | gzip -9 -c > your-pkg-M.N.tar.gz
1 tar chof - "$tardir" | bzip2 -9 -c > your-pkg-M.N.tar.bz2
1
1 However, if the hierarchy you are archiving and compressing is larger
1 than a couple megabytes, and especially if you are using a
1 multi-processor system with plenty of memory, then you can do much
1 better by reading the directory contents only once and running the
1 compression programs in parallel:
1
1 tardir=your-pkg-M.N
1 tar chof - "$tardir" \
1 | tee >(gzip -9 -c > your-pkg-M.N.tar.gz) \
1 | bzip2 -9 -c > your-pkg-M.N.tar.bz2
1
1 If you want to further process the output from process substitutions,
1 and those processes write atomically (i.e., write less than the system’s
1 PIPE_BUF size at a time), that’s possible with a construct like:
1
1 tardir=your-pkg-M.N
1 tar chof - "$tardir" \
1 | tee >(md5sum --tag) > >(sha256sum --tag) \
1 | sort | gpg --clearsign > your-pkg-M.N.tar.sig
1
1 An exit status of zero indicates success, and a nonzero value
1 indicates failure.
1